COleScript::Compile hook not working

[baxitaurus]

Hi guys, first of all thanks alot for your excellent work.

I'm having issues in getting the COleScript::Compile hook defined in insn/vbscript.yml to work properly, and that's confirmed by the DEBUG:Error resolving function vbscript!COleScript_Compile through our custom callback log messages I see in the analysis.log file.

I followed the steps described in this article to set up my hook, since my C:\Windows\System32\vbscript.dll has a different sha256 (9cb3ace7916fbe3876970f58870a6635f32ebeb0ab4aecece7a96be31434b2eb) I had to change both the offset of the PE timestamp and the one of the function inside the DLL, while I wasn't able to figure out what does stack: 56 refers to (how should be set this parameter?). Below you can see my definition of the hook:

   COleScript_Compile:                                                                                                                                                                                            
     module: vbscript
     offsets:
       0x512080e8:
         bitmode: 32
         offset: 0x12af0
         register: r8
         stack: 56
         logging:
        - u script r8
        - u type stk0

Despite of this, the hook seems not working yet. Is there some useful documentation about this "special" hooks? I can't find nothing but that article on the web, and I have no idea what is wrong with the above definition.

I'm working with a Windows 7 Professional x64 (SP1) VM and the sample I'm using for tests is a simple "Hello World" VBS script. My version of vbscript.dll is 5.8.9200.16521.

(I've tried with bitmode: 64 too)

Edit:
I checked the steps I followed to reach the above definition on the same vbscript.dll version used in the article, and they seems to be correct since I can obtain the same offsets used in the production version of this hook. What am i missing?

[jbremer]

Nice that you're trying to make this work :-) My first assumption would be: you looked at the 64-bit .dll, but wscript.exe loads the 32-bit .dll (or the other way around, but I think that's less likely). So in that case - different dll with different PE timestamp etc.

[baxitaurus]

Thanks for your reply @jbremer, but actually I did look at the 32-bit version of vbscript.dll because I saw that wscript was loading it (through IDA), so the offsets I used in the hook definition are referred to the 32-bit version 😄

Looking at the insn/flash.yml I saw an interesting comment:

  5 BaseExecMgr_setNative:
  6   module: flash32_20_0_0_228
  7   init: flash_init
  8   offsets:
  9     0x565123f2:
 10       # The BaseExecMgr::setNative method starts at 0x6d7900, however, it has
 11       # to process a flag in order to determine the real native address of the
 12       # method. We just await this computation before fetching it.
 13       offset: 0x705230
 14       stack: 4 8
 15       logging:
 16         - S method_name length, name
 17         - p offset flash_module_offset(stk1)
 18       pre: |
 19           uint32_t length = 0; const char *name;
 20           name = flash_get_method_name(stk0, &length);

I was wondering if something similar should be done with the new definition of the COleScript::Compile function in the version of vbscript.dll i'm working on. Maybe here the start address of the function is not the "real" start address of the function?

Below i screenshot of the function in the DLL used in the article (the hook definition in the master branch refers to it):

And here you can find "my" version of it:

[jbremer]

Well, all those screenshots show 64-bit x86, so ;-)

[baxitaurus]

Ok, I do have to study the definitions of 32/64 bit and x86/x64, I admit it 😆 Despite of this I'm sure the DLL that wscipt is loading is that one (under C:\Windows\System32), or at least this is what I see with WinDBG:

So the problem here should be not the DLL I'm looking at, am i wrong?