Friday, June 23, 2017

Hyper-V sockets and AfdConnect

  When we use sockets and call CONNECT method, it calls mswsock!WSPConnect. Then mswsock!WSPConnect calls NtDeviceIoControlFile. NtDeviceIoControlFile has following structure:
NTSTATUS WINAPI NtDeviceIoControlFile(
 _In_  HANDLE           FileHandle, - point to \Device\Afd
 _In_  HANDLE           Event,
 _In_  PIO_APC_ROUTINE  ApcRoutine, - 0
 _In_  PVOID            ApcContext, - 0
 _Out_ PIO_STATUS_BLOCK IoStatusBlock,- 0036F4C0
 _In_  ULONG            IoControlCode – 00012007 - IOCTL code
 _In_  PVOID            InputBuffer,- 0036F4E8
 _In_  ULONG            InputBufferLength, 30
 _Out_ PVOID            OutputBuffer,- 0
 _In_  ULONG            OutputBufferLength - 0
);
NtDeviceIoControlFile goes to execute of afd!AfdConnect function. It analyze InputBufferLength and, if it above AfdStandardAddressLength constant (eq 0x1C in Windows Server 2016) function, calls nt!ExAllocatePoolWithTagPriority
PVOID ExAllocatePoolWithTagPriority(
 _In_ POOL_TYPE        PoolType,- NonPagedPoolNx
 _In_ SIZE_T           NumberOfBytes,- (InputBufferLength-0xC)
 _In_ ULONG            Tag,- AfdR
 _In_ EX_POOL_PRIORITY Priority - LowPoolPriority
);

If we change InputBufferLength parameter of ntdll!NtDeviceIoControlFile from 0x30 to 0x4FFFFFFF before execution, we get:
kd> k
# Child-SP          RetAddr           Call Site
00 ffff8b80`7489c678 fffff805`53c46d9f nt!ExAllocatePoolWithTagPriority
01 ffff8b80`7489c680 fffff803`57293180 afd!AfdConnect+0x36f
02 ffff8b80`7489c820 fffff803`57292064 nt!IopSynchronousServiceTail+0x1a0
03 ffff8b80`7489c8e0 fffff803`572919e6 nt!IopXxxControlFile+0x674
04 ffff8b80`7489ca20 fffff803`56fd5493 nt!NtDeviceIoControlFile+0x56
05 ffff8b80`7489ca90 00000000`52dd222c nt!KiSystemServiceCopyEnd+0x13

kd> r
rcx=0000000000000200
rdx=000000004ffffff3 – buffer size
r8=0000000052646641  
r9=0000000000000000

kd> !poolused 2 AfdR
Sorting by NonPaged Pool Consumed
              NonPaged                  Paged
Tag     Allocs         Used     Allocs         Used
AfdR         6   1 342 178 160          0            0    Afd remote address buffer , Binary: afd.sys
TOTAL         6   1 342 178 160          0            0

Then user buffer (InputBuffer parameter) will be copy to allocated kernel pool by memove. If you don’t allocate user buffer page fault exception may be generated during of copy, especially if buffer size was set to hundreds of megabytes (exception will be handled by afd!AfdExceptionFilter). This kernel pool is freed by nt!ExFreePoolWithTag at the end of afd!AfdConnect.
It was tested with Hyper-V Sockets, but probably, it will work with standard TCP\IP sockets too.
I believe, that LowPoolPriority parameter of nt!ExAllocatePoolWithTagPriority will not allow operation system to hang because of limited kernel pool size.

Tested on afd version:

WINDBG>lmvm afd
   Image path: \SystemRoot\system32\drivers\afd.sys
   Image name: afd.sys
   Timestamp:        Sat Oct 15 06:53:45 2016 (5801A849)

Saturday, June 17, 2017

Hyper-V Sockets and PnP


How does Powershell Direct session work internally? When application, which uses Hyper-V socket, makes CONNECT call, root partition sends message to guest OS using winhvr!WinHvPostMessage:
kd> !dc @rdx – before vmcall (part of message body)
#227b36000 00000001 00000000 00000001 000000c4 ................
#227b36010 00000001 00000000 999e53d4 4c3e3d5c .........S..\=>L
#227b36020 d0be7987 e156c06e

If we look at the parameters of vmicvmsession, we find out, that it has trigger for start.


After the message was delivered to guest OS, vmbus driver added new interface:

kd> kcn
# Call Site
00 nt!IoRegisterDeviceInterface
01 Wdf01000!Mx::MxRegisterDeviceInterface
02 Wdf01000!FxDeviceInterface::Register
03 Wdf01000!FxDeviceInterface::Register
04 Wdf01000!imp_WdfDeviceCreateDeviceInterface
05 vmbus!RootStartDeviceInterfaceByContext
06 hvsocket!VmbusTlXPartProcessNewConnection
07 vmbus!RootNotifyDeviceInterfaceArrival
08 Wdf01000!FxWorkItem::WorkItemHandler
09 Wdf01000!FxWorkItem::WorkItemThunk
0a nt!IopProcessWorkItem
0b nt!ExpWorkerThread
0c nt!PspSystemThreadStartup
0d nt!KiStartSystemThread

And PnP mechanism works:
kd> kcn
# Call Site
00 nt!PnpNotifyDeviceClassChange
01 nt!PnpDeviceEventWorker
02 nt!ExpWorkerThread
03 nt!PspSystemThreadStartup


Service trigger works and service starts. After that icsvc.dll configures socket and begins communication with Host OS.
It is interesting that we can change that GUID to GUID of another device. We can get list of services, that have trigger on DEVICE INTERFACE ARRIVAL event:

All Hyper-V guest services
Bluetooth Support Service
Windows Camera Frame Server
Human Interface Device Service
Geolocation Service
Microsoft Passport
Portable Device Enumerator Service
Sensor Service
Sensor Monitoring Service
Storage Service
Touch Keyboard and Handwriting (probably)

And if we point that GUID in CONNECT call

clsid_str = L"{53f56307-b6bf-11d0-94f2-00a0c91efb8b}";
CLSIDFromString(clsid_str, &ServiceID); //when you call CONNECT with that GUID Portable Device Enumerator Service will start in guest OS (even in shielded VM)

guest OS starts this service, because it believes, that triggered device was appeared in system.
One point: before do this you must add GUID to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Virtualization\GuestCommunicationServices as key in Host OS (need local admin rights)

It works on Shielded VM too (tested in Admin-Trusted mode), besides Hyper-V Powershell Direct Service – it not started

   
               It very interesting to see integration between Hyper-V sockets and PnP subsystem in Windows.
   

Powershell Direct (few internals). Part 2

According https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/manage/manage-windows-virtual-machines-with-powershell-direct Powershell Direct session needs Hyper-V administrator privilege for user, which start that session. Yes, it is true, if you use Enter-PSSession cmdlet, but if you will be use Hyper-V sockets natively you don’t need any privileges for it.