1) It is not permissible to pray for forgiveness or say "RIP" (which is basically a dua for peace and comfort in the hereafter) upon the death of a non Muslim. This is because those who die as Kuffars are promised eternal punishment in the hereafter, and praying for their forgiveness is forbidden. Allah says in the Quran:
"It is not (permissible) for the
Prophet and the believers to seek forgiveness for the Mushriks, even if they are kinsmen, after it became clear to them that they are the people of hell." (Surah Al Tawbah, Verse no. 113)
2) The phrase "inna lillahi wa inna ilaihi raji'un" does not contain any element of dua or istighfar. Rather, it simply expresses the reality that we all belong to Allah and that we will all return to Him. However, according to the Quran, this phrase is to be recited in times of hardship and misfortune. Therefore, there is some hesitation in reciting it upon the death of a non Muslim. It is therefore better to remain silent on this occassion and reflect on one's own hereafter.