I'm building a server based off of http://www.tyan.com/products/html/thunderk8sre.html. It's using 2 dual core opterons, 4 sata drives in raid 5, and 8 GB of memory. The memory is where the problem lies. When I have only 4 GB installed, it detects all PCI devices with no issues. When I install 8 GB, it can't detect PCI devices, namely the onboard ethernet devices. I've tried pretty much every type of memory hole setting available in the BIOS, from no memory hole to a 3 GB memory hole, hardware, software, hole and all permutations in between, to no avail. Has anyone out there had experience building a server like this, and if so did you run into a problem like this and figure out a way to solve it?
Searching for linux and 8GB seems to show that you're not the first experiencing trouble with that combination, especially on Tyan boards. As a temporary solution you could try and pass memsize=<something a bit less than 8GB> to the kernel at boot. Are you using a kernel compiled with support for large memory?
I'm using Redhat Enterprise 4 update 1 which uses kernel 2.6.9-22 I think (don't have access to the server computer at the moment). It is using the x86_64 version. I think Redhat backports fixes into older versions of the kernel though, since I read something about this release having fixes from newer kernels in it. I'm not sure if it is compiled with support for large memory, but I can see 7 of the 8 GB of the RAM in top, so I think it can see the mem ok. I think the missing 1 GB is gone because of the memory hole. The driver I am using for the card is compiled from the most recent source rpm provided by Tyan for the bcm5700 network card.