Accessing devices involves the kernel interacting with the IO devices via a device driver.
Device drivers can be linked into the kernel image and then are available as needed. Or they can be added into the kernel as a separate module and without rebooting the kernel.
The kernel then has access to that module and therefore the driver, but because it is in a module, the kernel image can stay lean and mean and only call on the module if and when you need the device.
However having said this, there are different levels of device driver support, and although not all the types are currently supported with Linux, it seems to be pointing that way development wise. There are also limits to the number of device drivers that the kernel can support. We will mention all of the options:
No support - where the application program contains assembler code to access the IO ports of the device directly. (X-Windows handles graphic display) See iopl() and ioperm() where process given permissions to access a IO ports directly - the process would have to be running with root permissions at this time though.
Min support - here the kernel would see the hardware device and maybe even recognise it, but cannot deal with the IO interface of the device. Here the user apps access the device sequentially reading or writing a sequence of characters. (Currently serial and parallel port supported.)
Extend Support - the kernel would recognise the hardware device and can handle the IO interface directly, there may not even be a device file for the device.