*** This file contains messages I've culled off the net as well as previous discussions all of which have useful info on fixes that need to be added to ReactOS. messages are between five dashes on a line by themselves. If you implement the fix reffered to in a message, feel free to delete it from the file. Rex *** ----- Subject: [ros-kernel] Inside the Boot Process Date: Mon, 22 Mar 1999 22:05:47 +0100 From: Emanuele Aliberti For those working on the boot loader: in WinNt Magazine november 1998 issue (http://www.winntmag.com/) there is a detailed description, by Mark Russinovich, of the rôle the MBR, NTLDR, boot.ini, ntdetect.com... play in the boot process ("Inside the Boot Process, Part 1"). ----- Yes with DPCs, KeDrainDpcQueue should go to HIGH_LEVEL because it needs to synchronize with KeInsertDpcQueue. Also the idle thread should run at DISPATCH_LEVEL and regularly drain the dpc queue, that way if an irq happens and the dpc can't be executed immediately it will be executed as soon as the processor is idle rather than waiting for the next timer tick ----- About the console driver, I think it might be quite useful to have a simple way for apps to print to the screen for debugging. But when the kernel is more stable, console handling should be moved to user level because console printing needs to know about windows and so on which can only be done at user level. ----- Subject: Re: IMSAMP-how to avoid rebooting? Date: 9 Nov 1998 00:40:32 -0000 From: Charles Bryant Newsgroups: comp.os.ms-windows.programmer.nt.kernel-mode References: 1, 2 , 3 , 4 In article , David C. wrote: >The reason it won't unload when something is bound to it is the same >reason you can't unload any other driver that has an open client. If >you install any driver, and have a user program (or another driver) open >a handle to it, and then give the "net stop" command to unload it, >you'll find that the unload will be delayed until the user program >closes its handle. When developing a driver I found this to be a considerable nuisance. Frequently a bug would leave an IRP stuck in the driver and I couldn't unload and reload a fixed version. While reading NTDDK.H I found a suspicious constant and discovered that the Flags field in the device (the one which you OR in DO_BUFFERED_IO or DO_DIRECT_IO) has a bit called DO_UNLOAD_PENDING. By experiment I confirmed that this bit is set when you do 'net stop', so a driver can check it periodically (e.g. from a timer DPC every ten seconds) and cancel all queued IRPs if it is found to be set. Since this is not documented anywhere that I can find, it might be unwise to rely on it for production code, but it is very useful for debugging. Maybe someone with internals knowledge can comment on the reliability of it. ----- Subject: Re: Kernel bugs Date: Fri, 23 Oct 1998 12:08:36 -0700 From: rex To: Jason Filby References: 1 Jason Filby wrote: > Hi, > > Ok -- here's most of what I get when I press a key: > > Page fault detected at address 1fd4 with eip c042f794 > Recursive page fault detected > Exception 14(2) > CS:EIP 20:c042f794 > > Rex -- do you know of anyway to find out which function in what file > is causing the exception? I know that for problems in the kernel, you > just look in the ntoskrnl\kernel.sym file and find the EIP value which > matches the one given in the exception debug text. But what about > modules? How can we track exceptions that occur in functions in modules? > I know this is a little belated, but I thought I'd take astab at answering this anyway. add an option to the makefile for the module to generate a listing file with symbol information. Then, on a boot test, note the address that the module is loaded at, and subtract this from the EIP value. add any offset used in the module link specification (I dont think there currently is one), and look for the last symbol with a lower address offset. Brian, I have an idea on how to make this exception dump information a little more useful. We should have the load information for the load modules in memory somewhere. Perhaps the exception dump could check offending addresses to see if they lie in the kernel or in a module, and if they lie in a module the proper offset could be subtracted and this number could be displayed seperately. If I get a chance today, I'll make this change and send it to ya. Rex. ----- Subject: Re: Question on "Sending buffers on the stack to asynchronous DeviceIoControl with buffered I/O" Date: Mon, 16 Nov 1998 11:24:57 -0800 From: "-Paul" Organization: Microsoft Corp. Newsgroups: microsoft.public.win32.programmer.kernel, comp.os.ms-windows.programmer.nt.kernel-mode References: 1 Radu, I post the following information occassionally for questions such as yours. I hope it helps. -Paul Here is an explanation of buffers and DeviceIoControl. First, here are the parameters, BOOL DeviceIoControl( HANDLE hDevice, // handle to device of interest DWORD dwIoControlCode, // control code of operation to perform LPVOID lpInBuffer, // pointer to buffer to supply input data DWORD nInBufferSize, // size of input buffer LPVOID lpOutBuffer, // pointer to buffer to receive output data DWORD nOutBufferSize, // size of output buffer LPDWORD lpBytesReturned, // pointer to variable to receive output byte count LPOVERLAPPED lpOverlapped // pointer to overlapped structure for asynchronous operation ); METHOD_BUFFERED user-mode perspective lpInBuffer - optional, contains data that is written to the driver lpOutBuffer - optional, contains data that is read from the driver after the call has completed lpInBuffer and lpOutBuffer can be two buffers or a single shared buffer. If a shared buffer, lpInBuffer is overwritten by lpOutBuffer. I/O Manager perspective examines nInBufferSize and nOutBufferSize. Allocates memory from non-paged pool and puts the address of this pool in Irp->AssociatedIrp.SystemBuffer. The size of this buffer is equal to the size of the larger of the two bufferes. This buffer is accessible at any IRQL. copies nInBufferSize to irpSp->Parameters.DeviceIoControl.InputBufferLength copies nOutBufferSize to irpSp->Parameters.DeviceIoControl.OutputBufferLength copies contents of lpInBuffer to SystemBuffer allocated above calls your driver Device Driver perspective you have one buffer, Irp->AssociatedIrp.SystemBuffer. You read input data from this buffer and you write output data to the same buffer, overwriting the input data. Before calling IoCompleteRequest, you must - set IoStatus.Status to an approriate NtStatus - if IoStatus.Status == STATUS_SUCCESS set IoStatus.Information to the number of bytes you want copied from the SystemBuffer back into lpOutBuffer. I/O Manager Completion Routine perspective looks at IoStatus block, if IoStatus.Status = STATUS_SUCCESS, copies the number of bytes specified by IoStatus.Information from Irp->AssociatedIrp.SystemBuffer into lpOutBuffer completes the request METHOD_IN_DIRECT user-mode perspective lpInBuffer - optional, contains data that is written to the driver. This buffer is used in the exact same fashion as METHOD_BUFFERED. To avoid confusion, mentally rename this buffer to lpControlBuffer. This is typically a small, optional buffer that might contain a control structure with useful information for the device driver. This buffer is smal and is double buffered. lpOutBuffer - NOT OPTIONAL, This LARGE buffer contains data that is read by the driver. To avoid confusion, mentally rename this buffer to lpDataTransferBuffer. This is physically the same buffer that the device driver will read from. There is no double buffering. Technically, this buffer is still optional, but since you are using this buffering method, what would be the point??? I/O Manager perspective If lpInBuffer exists, allocates memory from non-paged pool and puts the address of this pool in Irp->AssociatedIrp.SystemBuffer. This buffer is accessible at any IRQL. copies nInBufferSize to irpSp->Parameters.DeviceIoControl.InputBufferLength copies nOutBufferSize to irpSp->Parameters.DeviceIoControl.OutputBufferLength copies contents of lpInBuffer to SystemBuffer allocated above So far this is completely identical to METHOD_BUFFERED. Most likely lpInBuffer (mentally renamed to lpControlBuffer) is very small in size. For lpOutBuffer (mentally renamed to lpDataTransferBuffer), an MDL is allocated. lpOutBuffer is probed and locked into memory. Then, the user buffer virtual addresses are checked to be sure they are readable in the caller's access mode. The MDL is address is stored in Irp->MdlAddress. Your driver is called. Device Driver perspective The device driver can read the copy of lpOutBuffer via Irp->AssociatedIrp.SystemBuffer. Anything written by the device driver to this buffer is lost. The I/O Manager does not copy any data back to the user-mode buffers as it did in the completion routine for METHOD_BUFFERED. Art Baker's book is wrong in this respect (page 168, "data going from the driver back to the caller is passed through an intermediate system-space buffer" and page 177, "When the IOCTL IRP is completed, the contents of the system buffer will be copied back into the callers original output buffer". The device driver accesses the Win32 buffer directly via Irp->MdlAddress. The driver uses whatever Mdl API's to read the buffer. Usually, this buffer is to be written to some mass storage media or some similar operation. Since this is a large data transfer, assume a completion routine is required. mark the Irp pending queue it return status pending Device Driver Completion Routine perspective standard completion routine operations set IoStatus.Status to an approriate NtStatus IoStatus.Information is not needed completete the request I/O Manager Completion Routine perspective standard I/O Manager completion routine operations unmap the pages deallocate the Mdl complete the request METHOD_OUT_DIRECT user-mode perspective lpInBuffer - optional, contains data that is written to the driver. This buffer is used in the exact same fashion as METHOD_BUFFERED. To avoid confusion, mentally rename this buffer to lpControlBuffer. This is typically a small, optional buffer that might contain a control structure with useful information for the device driver. This buffer is smal and is double buffered. lpOutBuffer - NOT OPTIONAL, This LARGE buffer contains data that is written by the driver and read by the wer-mode application when the request is completed. To avoid confusion, mentally rename this buffer to lpDataTransferBuffer. This is physically the same buffer that the device driver will write to. There is no double buffering. Technically, this buffer is still optional, but since you are using this buffering method, what would be the point??? I/O Manager perspective If lpInBuffer exists, allocates memory from non-paged pool and puts the address of this pool in Irp->AssociatedIrp.SystemBuffer. This buffer is accessible at any IRQL. copies nInBufferSize to irpSp->Parameters.DeviceIoControl.InputBufferLength copies nOutBufferSize to irpSp->Parameters.DeviceIoControl.OutputBufferLength copies contents of lpInBuffer to SystemBuffer allocated above So far this is completely identical to METHOD_BUFFERED. Most likely lpInBuffer (mentally renamed to lpControlBuffer) is very small in size. For lpOutBuffer (mentally renamed to lpDataTransferBuffer), an MDL is allocated. lpOutBuffer is probed and locked into memory. Then the user buffer's addresses are checked to make sure the caller could write to them in the caller's access mode. The MDL is address is stored in Irp->MdlAddress. Your driver is called. Device Driver perspective The device driver can read the copy of lpOutBuffer via Irp->AssociatedIrp.SystemBuffer. Anything written by the device driver to this buffer is lost. The device driver accesses the Win32 buffer directly via Irp->MdlAddress. The driver uses whatever Mdl API's to write data to the buffer. Usually, this buffer is to be read from some mass storage media or some similar operation. Since this is a large data transfer, assume a completion routine is required. mark the Irp pending queue it return status pending Device Driver Completion Routine perspective standard completion routine operations set IoStatus.Status to an approriate NtStatus IoStatus.Information is not needed completete the request I/O Manager Completion Routine perspective standard I/O Manager completion routine operations unmap the pages deallocate the Mdl complete the request METHOD_NEITHER I/O Manager perspective Irp->UserBuffer = lpOutputBuffer; IrpSp->Parameters.DeviceIoControl.Type3InputBuffer = lpInputBuffer; No comments here. Don't use METHOD_DIRECT unless you know what you are doing. Simple rule. If your IOCtl involves no data transfer buffers, then METHOD_NEITHER is the fastest path through the I/O Manager that involves an Irp. Final Comment Don't touch Irp->UserBuffer. This is a bookmark for the I/O Manager. Two major problems can occur. 1 - page fault at high IRQL, or 2 - you write something to Irp->UserBuffer and the I/O Manager overwrites you in its completion routine. File systems access Irp->UserBuffer, but FSD writers know all of the above and know when it is safe to touch Irp->UserBuffer. Radu Woinaroski wrote in message <364F8F6E.2434B010@scitec.com.au>... >Hello, > >I have a kernel-mode device driver that accepts a number of IoControl >commands that use buffered data transfer (METHOD_BUFFERED). > >A user mode API provides a higher level access then the DeviceIoControl >function. > >The function is implemented like that > >BOOL Something( > HANDLE hDevice , > int param1, > int param2, > DWORD * pReturn, > LPOVERLAPPED pOverlapped) >{ > // here a data buffer on the stack sent to asynchronous DeviceIoControl >call > int aDataIn[2]; > aDataIn[0] = param1; > aDataIn[1] = param2; > > return DeviceIoControl( > hDevice, > DO_SOMETHING_IO, > aDataIn, > sizeof(int)*2, > pReturn, > sizeof(DWORD), > pOverlapped); >} > >The aDataIn buffer will not exist after DeviceIoControl returns (and >when the I/O operation terminates). I know that for buffered IO the >input data buffer is copyed by de IOManager to a nonpaged-pool area >before passing the request to driver dispatch routine (DeviceControl). >At the point of calling the dispatch routine (DeviceControl) the driver >runs in the context of the calling thread so DeviceIoControl hasn't >returned yet (?? or so I think) so aDataI n will still be valid at the >time IOManager copyes it to its buffer. So, this apears to work ok (at >least in my opinion). > >Does I/O Manager use the Input buffer from the call to the Win32 >DeviceIoControl any where else after the first copy ? > >Is there any reason why this approach (passing a buffer on the stack to >a asynchronous DeviceIoControl that uses buffered I/O) wouldn't work ? > >Allocating buffers from heap and deleting them on IO completion while >managing asynchronous IO seems too much work ;-) . > >Thanks in advance for your opinions >Radu W. > >-- >Radu Woinaroski >Scitec >Sydney, Australia >Radu.Woinaroski@scitec.com.au ----- Subject: Re: PCI ISR problem Date: Fri, 20 Nov 1998 18:04:48 GMT From: jeh@cmkrnl.com (Jamie Hanrahan) Organization: Kernel Mode Systems, San Diego, CA Newsgroups: comp.os.ms-windows.programmer.nt.kernel-mode References: 1 On Thu, 19 Nov 1998 15:46:13 -0600, Eric Gardiner wrote: >I'm having problems with NT4 not hooking the interrupt line indicated by >a PCI device. Here's what I'm doing: > >1) Enumerating the PCI buses on the system (using HalGetBusData) until >I find my device. >2) Once my device is found, I read the "Interrupt Line Register" in the >device's PCI config space to determine what interrupt level to pass to >HalGetInterruptVector. Whups! No. Call HalAssignSlotResources and look at the returned CM_RESOURCE_LIST to find the vector, level, port addresses, etc., for your device. (Then pass the returned CM_RESOURCE_LIST to ExFreePool.) See Knowledge Base article Q152044. --- Jamie Hanrahan, Kernel Mode Systems, San Diego CA (jeh@cmkrnl.com) Drivers, internals, networks, applications, and training for VMS and Windows NT NT kernel driver FAQ, links, and other information: http://www.cmkrnl.com/ Please post replies, followups, questions, etc., in news, not via e-mail. ----- Subject: Re: IRP canceling Date: Mon, 23 Nov 1998 09:05:47 -0500 From: Walter Oney Organization: Walter Oney Software Newsgroups: comp.os.ms-windows.programmer.nt.kernel-mode References: 1 Seol,Keun Seok wrote: > But, if the IRP was the CurrentIrp of the Device Object, > the Driver's Start I/O routine will try to process the IRP. > In the DDK help, the Start I/O routine MUST check the current IRP's > Cancel bit. > If set, Start I/O routine must just return. > > But I think that the IRP already completed should not be accessed. You're absolutely right. I recommend the following code in a standard StartIo routine to avoid the problem you point out: VOID StartIo(PDEVICE_OBJECT DeviceObject, PIRP Irp) { KIRQL oldirql; IoAcquireCancelSpinLock(&oldirql); if (Irp != DeviceObject->CurrentIrp || Irp->Cancel) { IoReleaseCancelSpinLock(oldirql); return; } else { IoSetCancelRoutine(Irp, NULL); IoReleaseCancelSpinLock(oldirql); } . . . } This dovetails with a standard cancel routine: VOID CancelRoutine(PDEVICE_OBJECT DeviceObject, PIRP Irp) { if (DeviceObject->CurrentIrp == Irp) { IoReleaseCancelSpinLock(Irp->CancelIrql); IoStartNextPacket(DeviceObject, TRUE); } else { KeRemoveEntryDeviceQueue(&DeviceObject->DeviceQueue, &Irp->Tail.Overlay.DeviceQueueEntry); IoReleaseCancelSpinLock(Irp->CancelIrql); } Irp->IoStatus.Status = STATUS_CANCELLED; Irp->IoStatus.Information = 0; IoCompleteRequest(Irp, IO_NO_INCREMENT); } You need to remember that the C language specification requires that evaluation of boolean operators short circuit when the result is known. So, if StartIo discovers that the Irp it got as an argument is not the same as CurrentIrp, it will not attempt to evaulate Irp->Cancel. Now, as to why this works: StartIo gets called either by IoStartPacket or IoStartNextPacket. Each of them will grab the cancel spin lock and set CurrentIrp, then release the spin lock and call StartIo. If someone should sneak in on another CPU and cancel this very same IRP, your cancel routine will immediately release the spin lock and call IoStartNextPacket. One of two things will then happen. IoStartNextPacket may succeed in getting the cancel spin lock, whereupon it will nullify the CurrentIrp pointer. If another IRP is on the queue, it will remove it from the queue, set CurrentIrp to point to this *new* IRP, release the spin lock, and call StartIo. [You now have two instances of StartIo running on two different CPUs for two different IRPs, but it's not a problem because they won't be able to interfere with each other.] Meanwhile, your original instance of StartIo gets the cancel spin lock and sees that CurrentIrp is not equal to the IRP pointer it got as an argument, so it gives up. The second way this could play out is that StartIo gets the cancel lock before IoStartNextPacket does. In this case, CurrentIrp is still pointing to the IRP that's in the process of being cancelled and that StartIo got as an argument. But this IRP hasn't been completed yet (the CPU that's running your cancel routine is spinning inside IoStartNextPacket and therefore hasn't gotten to calling IoCompleteRequest yet), so no-one will have been able to call IoFreeIrp to make your pointer invalid. People may tell you that you should be using your own queues for IRPs so you can avoid bottlenecking the system on the global cancel spin lock. That's true enough, but doing it correctly with Plug and Play and Power management things in the way is gigantically complicated. There's a sample in the NT 5 beta-2 DDK called CANCEL that shows how to manage your own queue if you don't worry about PNP and POWER. I hear tell of an upcoming MSJ article by a Microsoft developer that may solve the complete problem. ----- The END.