Thursday, September 20, 2007

GetOpenFileName() And Some Deep crashes..

I think most of all windows programmers are familiar with GetOpenFileName() API (in MFC CFileDialog class).This api popups a open file dialog letting users to select the file they want.
Okay , I am going into the matter.
Try opening notepad.
1. click open from menu , now you can see open file dialog. Navigate to Desktop
2  move the mouse pointer around some file till tool tip visible.
3. Now cancel the dialog (press cancel or ESC)
4. Try to repeat the same thing (make sure again you go to desktop) , you will be awaited with a crash. Notepad disappears.


Interestingly This happens only with Desktop. I don't know why . So you might ask me how it mess up with the GetOpenFileName. Yes it relates , try calling GetOpenFileName in win32 App (not MFC). You can see the same behavior , your application crashes..


This is Bug in windows XP ( Service pack??? , any oher Version ??? ) .. i don't about any other versions.
If you debug the application you can't see any stack trace . This also happens with the GetSaveFileName api also.
For calling GetOpenFileName you need initialize COM , by calling CoInitialize(0) in the first of the program and call CoUnitinialize in the end of the program. Then it should work. The interesting thing is that in MSDN there is no information about this.


If you call like this as show below , still you can expect the crash.
CoInitialize(0);
GetOpenFileName();
CoUnitialize()

I don't know the exact reason . But my guess is that the GetOpenFileName() / SaveFileName() creats 4 threads .Even you close the dialog these thread exists ,  and these threads might call some com calls , after your CoUnitialize() . That may cause some problems. The most important this is that , There is a chance to crash every application ,  that uses GetOpenOpenFile/GetSaveFile dialog (and not initializing COM) . Seriously..

14 comments:

msnkd said...

Yes,It is reproducable... Very funny.

Thomas said...

Are  you able to reproduce it in the latest SP2?  I can\'t.  But I can produce similar problem in my program.  Every time I right click on a folder in the file open dialog, it will crash  (when pageheap is enabled).  If I disable pageheap, it\'s fine.
 
Do you have any new clue?

Thomas said...

Are  you able to reproduce it in the latest SP2?  I can\'t.  But I can produce similar problem in my program.  Every time I right click on a folder in the file open dialog, it will crash  (when pageheap is enabled).  If I disable pageheap, it\'s fine.
 
Do you have any new clue?

Thomas said...

Are  you able to reproduce it in the latest SP2?  I can\'t.  But I can produce similar problem in my program.  Every time I right click on a folder in the file open dialog, it will crash  (when pageheap is enabled).  If I disable pageheap, it\'s fine.
 
Do you have any new clue?

Thomas said...

i m very sorry for the multiple posts.  It\'s a server error in live.com and i reposted.

Thomas said...

i m very sorry for the multiple posts.  It\'s a server error in live.com and i reposted.

Thomas said...

i m very sorry for the multiple posts.  It\'s a server error in live.com and i reposted.

Thomas said...

hey, what\'s wrong?  this time i hit "ADD" button only once.   but my post appeared 3 times!  Now watch this post.  I think it will be posted 3 times too.

Thomas said...

hey, what\'s wrong?  this time i hit "ADD" button only once.   but my post appeared 3 times!  Now watch this post.  I think it will be posted 3 times too.

msnkd said...

zt , i couldn\'t check it on sp2. but i guess the problem is with CoInitialize. did u checke d it by adding CoInitialize ?

Thomas said...

Yes I tried CoInitialize and it did not help.  However, I did find out the source of heap violation:
 
1. nvshell.dll located in windows system32.  This is from NVidia graphics card.
2. fpshx.dll located in C:\\Program Files\\F-Secure\\common.  This is from F-Secure Anti-Virus.
 
After i rename both files, the Open File Dialog no longer crashes with MS WordPad when the MS PageHeap is enabled.

Thomas said...

Yes I tried CoInitialize and it did not help.  However, I did find out the source of heap violation:
 
1. nvshell.dll located in windows system32.  This is from NVidia graphics card.
2. fpshx.dll located in C:\\Program Files\\F-Secure\\common.  This is from F-Secure Anti-Virus.
 
After i rename both files, the Open File Dialog no longer crashes with MS WordPad when the MS PageHeap is enabled.

Thomas said...

Yes I tried CoInitialize and it did not help.  However, I did find out the source of heap violation:
 
1. nvshell.dll located in windows system32.  This is from NVidia graphics card.
2. fpshx.dll located in C:\\Program Files\\F-Secure\\common.  This is from F-Secure Anti-Virus.
 
After i rename both files, the Open File Dialog no longer crashes with MS WordPad when the MS PageHeap is enabled.

msnkd said...

Good to hear you solved it, In my system i have nvshell.Dll but not pshx.dll. but tt works after calling CoInitialize function.These type of bugs are really painful.good luck