To make your kernel as small as possible, and to get rid of everything you don't need, make everything except a few important things in the kernel modular. You need to have a few obvious things like IDE/SCSI access and ext2 support built in to the kernel.
Create a customized monolithic kernel tuned to your hardware.
cd /usr/src/linux; make xconfig

Set Processor family to PPro/6x86MX for Pentium II/III systems.
Enable MTRR for PentiumPro/II/III and newer AMD K6-2/3 systems.
Only enable Symmetric multiprocessing if you are using a multiprocessor box.
Disable Advanced Power Management if you are running a server configuration.
Disable chipset/bugfix support for all chipsets you DON'T have.
Example: CMD640 and RZ1000
Enable Generic PCI bus-master DMA support and Use DMA by default
Enable any custom support for chipsets you DO have.
Example: VIA82C586 for VIA VP2/3 and MVP3 motherboards.
Enable RAID-0 (striping) if you have multiple identical disks.
Disable Kernel/User netlink socket
Enable only the SCSI device types you actually have.
Example: SCSI disk, but not tape/CD-ROM/generic
Disable Probe all LUNs unless you need it.
Disable Verbose SCSI error reporting
Disable SCSI logging facility
Enable ONLY the specific SCSI controller you have.
Example: Adaptec AIC7xxx
Test and Tune the Maximum number of commands per LUN
Disable ALL Network device support that you don't strictly require.
Set the Maximum number of Unix98 PTYs to 128
Disable Watchdog Timer Support
Disable all Filesystems
EXCEPT: ISO 9660 CDROM, Second extended fs,/dev/pts, and /proc support.
Disable all Network File Systems
If you need NFS server support edit the source and #undef NFS_PARANOIA
Disable all Console drivers
EXCEPT: VGA text console
Disable Enhanced Real Time Clock Support
Disable all non-essential serial and mouse device support.
Disable all Video for Linux
Disable all non-SCSI/IDE/ATAPI CD-ROMs
Disable all ISDN
Disable all Plug and Play