Background
I am working on a KMDF driver for a PCIe device we are developing. One of the features required is to perform autonomous P2P transfer of data between the devices and I am trying to figure out a reasonable way to implement the API for this. All devices use the same custom KMDF driver (they can dynamically change between sender or receiver) and are simply appearing as individual device objects for that same driver.
The devices themselves (FPGAs) have their own DMA engines and bus-mastering capabilities. For testing purposes, I implemented a very simple way of controlling this using a couple of IOCTL codes. For the receiving device, an IOCTL for the driver returns a bus-relative address of the receive buffer in that FPGA. For the sending device a second IOCTL is implemented to which the address of the target device is provided. The sender then performs DMA operations to this target address.
However as we move out of development into a release oriented driver, this approach seems highly unsafe and insecure. It basically provides a kernel API that can instruct the device to write to arbitrary memory addresses that are provided by the user land (user software relays the address information from one instance to another).
Question
There must be a better way of doing this. The first thought was some sort of encryption or signing of the address, but this feels like just patching over a poor design to make it feel safer.
I imagine a better way is to keep the addressing completely in kernel land, and instead return some form of unique target ID for the receiver device to user land. When the sender driver instance is provided the target ID, it would then need to find some way of looking up the actual bus address for that device. This would require the different device device instances to be able to communicate with each other to perform the address lookup.
Unfortunately I’m not familiar enough with the KMDF framework to figure out an approprate way of implementing this communication, so some pointers would be appreciated. Specifically the bits I’m missing:
-
Assigning a unique ID to each instance of the driver.
- I was thinking either some sort of shared counter in the driver that tracks how many instances have been created,
- Alternatively if there is a way of looking up say the PCIe bus-device-function address of the receiver and using that as an ID.
- Maybe I’m over thinking it and the framework might already have some ID for each instance that I could use directly.
-
Communication between different instances of the driver to perform address lookup.
- I could add an internal API to the driver, or store address information in each device context, but I would need to find a way to find the device context from an ID.
Ideally I would like this to work with KMDF framework versions as old as 1.11 as we still have a system connected to an old Win 7 machine (non-networked) that can’t be updated yet for other reasons. But I would be OK with it working with version 1.15.
If possible I’d like to avoid having to deal with playing with with the underlying WDM objects and stick with well documented KMDF APIs.