If your sister's mental stability has deteriorated to the point where she is unable to comprehend the obligation of prayers and fasting, she is not obligated to perform them according to Shariah. Therefore, it is not obligatory to provide fidyah for her missed prayers and fasts.
However, if your sister is sane enough to understand that prayers and fasting are "farz", but ends up making mistakes in her prayers and fasting, her prayers cannot be compensated for by giving fidyah. Her family members should help her in praying. One family member should sit with her and guide her through the steps of salah, such as rukoo and sujood. Alternatively, the other women in the house should pray together and encourage her to follow along.
As for fasting, if your sister experiences some improvement in her mental state during the month of Ramadan (meaning she is not unconscious or mentally disturbed to the point of madness for the entire month), she must perform qaza for the fasts she missed. If she is unable to perform qaza in her lifetime and requests her relatives to give fidyah on her behalf in her will, it is obligatory to give the fidyah.
The amount to be given as the fidyah for one fast is 1.75 kg of wheat or money equivalent to the price of 1.75 kg of wheat.
۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔
دلائل:
بدائع الصنائع: (89/2، ط: دار الکتب العلمیہ)
والنائم بخلاف الجنون المستوعب فإن هناك في إيجاب القضاء حرجا لأن الجنون المستوعب قلما يزول بخلاف الإغماء، والنوم إذا استوعب لأن استيعابه نادر، والنادر ملحق بالعدم بخلاف الجنون فإن استيعابه ليس بنادر، ويستوي الجواب في وجوب قضاء ما مضى عند أصحابنا في الجنون العارض ما إذا أفاق في وسط الشهر، أو في أوله حتى لو جن قبل الشهر ثم أفاق في آخر يوم منه يلزمه قضاء جميع الشهر۔
الفتاوی الھندیۃ: (138/1، ط: دار الفکر)
مصل أقعد عند نفسه إنسانا فيخبره إذا سها عن ركوع أو سجود يجزيه إذا لم يمكنه إلا بهذا، كذا في القنية.
و فیھا ایضاً: (206/1، ط: ماجدیۃ)
فالشیخ الفانی الذی لا یقدر علی الصیام یفطر ویطعم لکل یوما مسکینا۔
فتاوی حقانیہ: (147/4)
Allah the Almighty knows best.
Darul ifta Al-Ikhlas, Karachi.