Who's planning a multisocket setup?

[Kayin]

Well, we know our good buddy Rubidium has that nice four-socket Opty setup, are you planning a dually or quad setup in the future?

I'm searching Ebay for a dual G5 setup for Kerykthea rendering, since it doesn't support clustering. Dual duals processes memory more efficiently, and I can get parts for those cheap off fleabay. Honestly, a lot of dually setups are cheap there, you should check it out if you're interested...

[ballz0r]

I wouldnt know what to look for.... or what to use it for when I got it.... but i like it

[Kayin]

Crunching/folding, rendering (Kerkythea is the Sketchup renderer, it's a free download) showing off, saying "I have x in my living room, what you got?", etc, etc...

After work I'll try to snap a pic of my case of PII Xeons in the closet. Yeah. A whole case. I used to work a lot with multisocket setups.

[Xilikon]

I used to have a 2x Pentium 166MMX setup in the past and it was a screamer in their time. I don't have any plans for a multi-socket machine since multi-cores is sufficient for me.

[ND40oz]

Plan on building a gainestown system in the next few months.

[ballz0r]

do you need to run any particular OS to get the most out of multi proc systems? also does an app need to be specially optimised to take advantage?

[Kayin]

Most run at least a 64-bit OS for the memory support, but there are specific server OSes for certain tasks, and to get some things to talk right takes certain OSes. Anything can be run on a server, but depending on threading, etc-either it allocates one task to one proc/core or it can devote them all-it's normally still faster. Remember, even though dual duals=1 quad, there's a pair of memory banks, and AMD and Intel have quad channel memory in servers. For AMD, fill up all the banks and get NUMA (http://en.wikipedia.org/wiki/Non-Uniform_Memory_Access), and the newer AMDs get ccNUMA which is even nicer.

AMD, with the point-to-point HT links, has been kicking the snot out of quad Intel for a while now. Only Tigerton seems to have a chance of dethroning it. NUMA is part of the reason it scales so well.

Keep the questions coming...

[rubidium]

do you need to run any particular OS to get the most out of multi proc systems? also does an app need to be specially optimised to take advantage?

I've had this addiction to multi-socket systems for 5 years now. My first build was a watercooled dual Opteron-280 rig with 16GB of RAM that's been running 24/7 with a 12% overclock since the start. It's actually a triple boot system: XP-64, Vista-64, Ubuntu-64, and all three OS's run just fine.

If you exceed 2 sockets, as I just have with my Project Erinyes build, Microsoft says :bird:, and won't give you access to the rest unless you buy their uber expensive Windows enterprize server editions. I'm currently playing around with a Server 2008 (x64) trial edition, modified to essentially appear and perform as a Vista workstation - with the exception of seeing all 4 of my sockets. However, I'm not so enamored with Windows that I actually want to pay that kind of money for an OS. I can build a whole other rig for that price. On the other hand, Ubuntu 8.10 installed in 10 minutes, gave me my 4 sockets/16 cores/64GB right from the get-go, doesn't have anywhere near the number of bullshit baggage system services running in the background like Windows, and seems to be up to meeting my needs in most other regards. Oh yea, and the price is right - FREE.

A 64-bit OS is de rigeur in my opinion for getting the most out of these builds, and as Kayin said NUMA is sweet.

As far as hardware is concerned, since the focus of multi-socket has long been the server market, there are a number of challenges (I won't say barriers) that need to be addressed. Most of those motherboards - especially the Tyan's - aren't set up with BIOS's that are overclock friendly. That isn't to say that overclocking can't be done. It's just that it's not as simple as going into the BIOS to adjust clock speeds, multipliers, and voltages as readily as you can with "enthusiast" boards, and a lot of the overclocking software tools (like Clockgen, etc) are only slowly catching up to the chipsets these boards are using. This is why I'm starting to get into understanding my chipset and how to do BIOS modding - to get the most out of these boards. It's a new challenge, and it will take an investment in time, but that's exactly why I'm into it.

Most "server" boards also use ECC registered memory, generally DDR2 - not exactly the thoroughbreds among contemporary RAM offerings. However, one can usually turn off ECC in the BIOS, and unlike the lack of BIOS options to tweak processor performance, BIOS's generally give you the ability to tighten timings. Then too, there's NUMA to help your bandwidth!

That being said, most of the chipsets on these big server boards are real ass kickers, capable of pushing lots of PCIe bandwidth which is great for graphics. My 4-socket board even supports SLI.

It's good to see multi-socket interest starting to perk up here ...
rubidium

[Kayin]

Used to run a dual Slot2 Xeon 733/133/2mb with 2GB of 1066 RDRAM. Tell me that wasn't an insane setup. Dual onboard SCSI U320 controllers and all...

Rubidium, shouldn't you run half the capacity and twice the sticks to speed up your memory?

[Ricey]

I want a 16 core quad processor system like Rubidium. I'm still shopping around.

[Cutless009]

I'm waiting until 2010 to build mine, apparently AMD is skipping the 8 core opty's and going straight to 12 cores (two inter-linked 6-core cpu's on a single chip).

http://news.softpedia.com/news/AMD-039-s-12-Core-Magny-Cours-Chip-Sports-Direct-Connect-Module-85235.shtml

I can wait that long before building my own personal super computer :)

[rubidium]

Yea the future is bright indeed. In my case, I'll at least be able to upgrade to 24 cores with just a processor swap out (Istanbul later in 09), before the socket 1207 is finally retired.

[sidewinder]

Two builds ago I put together a nice dual Xeon system with an Iwill DH800 board. It was one of the only solutions around at the time that would allow for xeons from the bios. I sure wouldn't mind building another one like yours, but the 8*** series chips require a bit of investment. Have you had any bug issues with 8.10, Rubidium?

[ND40oz]

If you exceed 2 sockets, as I just have with my Project Erinyes build, Microsoft says :bird:, and won't give you access to the rest unless you buy their uber expensive Windows enterprize server editions. I'm currently playing around with a Server 2008 (x64) trial edition, modified to essentially appear and perform as a Vista workstation - with the exception of seeing all 4 of my sockets. However, I'm not so enamored with Windows that I actually want to pay that kind of money for an OS. I can build a whole other rig for that price. On the other hand, Ubuntu 8.10 installed in 10 minutes, gave me my 4 sockets/16 cores/64GB right from the get-go, doesn't have anywhere near the number of bullshit baggage system services running in the background like Windows, and seems to be up to meeting my needs in most other regards. Oh yea, and the price is right - FREE.

You need to get MSDN or Technet, then you'll have access to all the OS versions.

That or run ESXi and virtualize your OSes.

[rubidium]

...
I sure wouldn't mind building another one like yours, but the 8*** series chips require a bit of investment. Have you had any bug issues with 8.10, Rubidium?

Gary is certainly right about the "investment", and it's an issue I thought a lot about before moving forward with the build. For me, the investment considerations required navigating a trade space involving money and patience. Back in the early fall of 2008 when I started the project design, a core strategy for me was to set myself up with fundamentals that would satisfy my needs well into the future: case, cooling system, power. After that, an important focal point in my thinking was the promise of a continuing (or accelerating) escalation in technology scale reduction (65nm->45nm->35nm->?) and core count per processor (4->6->8->12->?). Accordingly, the personal investment decision underscoring my choice of a 4-socket system centered less on dropping big money and more on exercising patience - by which I mean my willingness to accept being a step behind the state-of-the-art and not demanding the latest, top-end, most expensive processor just released last week. For example, just to get my rig up and going - but, nevertheless, with significant performance - I started out with 4x 8346HE's, which I got brand new for $50 each. Soon AMD is going to introduce Istanbul, and so I'm confident that later in the year I'll be making a significant upgrade to much higher performing Shanghai's when their prices drop like stones as what happened to the Barcelona's in the precious cycle.

Regarding Ubuntu, I've only started exploring 8.10 within the last couple to three weeks, and to date haven't experienced any major bugs. I've been running 8.04 (Hardy Heron) on my dual-socket rig (Oppy-280's) for a lot longer, and it's been pretty solid. With 8.04 and earlier releases, driving a Marvell Yukon ethernet controller was very problematic (fortunately I also have an nVidia NIC on that mobo which works just fine), and hopefully this has gotten fixed with 8.10. Also, back in October when I tried to transition from 8.04 to 8.10 using the update process within the OS, rather than do a completely virgin install, it took hours and ended in complete disaster! Fortunately, I had thoughtfully imaged my hard drive beforehand and completely recovered my original state 10 minutes later.

rubidium

[Burnout21]

If you had a budget of £300 what SMP plateform would you look for on ebay as a second hand beast?

currently rocking a Q6600, and tbh when it comes to rendering with maxwell it really gets hit in the nads.

I have been looking out for mATX 939 boards to build a multi node render farm, but i am trying to look at another way for number crunching power on a budget.

[ND40oz]

If that's your budget, go with a dual quad opty. You can get the quads dirt cheap if they have the tlb bug and memory is cheaper then the fb-dimms used on the intel side. Decent boards are cheaper too, only two Seaburg boards allow OCing in the bios and they start new at $500.

[Burnout21]

If that's your budget, go with a dual quad opty. You can get the quads dirt cheap if they have the tlb bug and memory is cheaper then the fb-dimms used on the intel side. Decent boards are cheaper too, only two Seaburg boards allow OCing in the bios and they start new at $500.

£300 is the budget for mobo and CPU's so a dual Quad opty system i think is out of my budget range! lol!

Just looking for a cheap number cruncher

[ND40oz]

£300 is the budget for mobo and CPU's so a dual Quad opty system i think is out of my budget range! lol!

Just looking for a cheap number cruncher

You'd be surprised, I've seen a pair of 1.9 ghz quads go for $120 on ebay...I'll see if I can find something for you.

Edit: here's an example: http://cgi.ebay.com/AMD-OS2346PALl4BGH-QUAD-CORE-2346HE-1-8GHZ-3RD-GEN-CPU_W0QQitemZ220375217388QQcmdZViewItemQQptZLH_Def aultDomain_0?hash=item220375217388&_trksid=p3286.m20.l1116

just keep your eye out

Here's a decent package if you're total budget can got up to 500 pounds or so.

http://cgi.ebay.com/ASUS-KFN32-D-SLI-SAS-MB-2-Opteron-Quad-Core-8347HE_W0QQitemZ300305454800QQcmdZViewItemQQptZLH_ DefaultDomain_0?hash=item300305454800&_trksid=p3286.c0.m14&_trkparms=72%3A1205%7C66%3A2%7C65%3A12%7C39%3A1%7C 240%3A1318%7C301%3A1%7C293%3A1%7C294%3A50

[Burnout21]

You'd be surprised, I've seen a pair of 1.9 ghz quads go for $120 on ebay...I'll see if I can find something for you.

Edit: here's an example: http://cgi.ebay.com/AMD-OS2346PALl4BGH-QUAD-CORE-2346HE-1-8GHZ-3RD-GEN-CPU_W0QQitemZ220375217388QQcmdZViewItemQQptZLH_Def aultDomain_0?hash=item220375217388&_trksid=p3286.m20.l1116

just keep your eye out

Here's a decent package if you're total budget can got up to 500 pounds or so.

http://cgi.ebay.com/ASUS-KFN32-D-SLI-SAS-MB-2-Opteron-Quad-Core-8347HE_W0QQitemZ300305454800QQcmdZViewItemQQptZLH_ DefaultDomain_0?hash=item300305454800&_trksid=p3286.c0.m14&_trkparms=72%3A1205%7C66%3A2%7C65%3A12%7C39%3A1%7C 240%3A1318%7C301%3A1%7C293%3A1%7C294%3A50

Holy shit, i could run an 8 core beast! let me see how much cash i've got, and i might just have to do that!

Does the Socket F plateform like standard DDR2 memory?

[BlackPumpkin]

I'm working on building a dual quad system myself. I got a board for $60 and am looking at either dual Barcs or dual Shanghais.

I've also got 2-3 Athlon MP boards, a dual 940 board, and a dual P3 board.

Duals are fun! :D

[Kayin]

Just where did you get a board for 60 bucks?

I gotta know, so long as it's not Fleabay.

[rubidium]

You'd be surprised, I've seen a pair of 1.9 ghz quads go for $120 on ebay...I'll see if I can find something for you.

Edit: here's an example: http://cgi.ebay.com/AMD-OS2346PALl4BGH-QUAD-CORE-2346HE-1-8GHZ-3RD-GEN-CPU_W0QQitemZ220375217388QQcmdZViewItemQQptZLH_Def aultDomain_0?hash=item220375217388&_trksid=p3286.m20.l1116

just keep your eye out

Here's a decent package if you're total budget can got up to 500 pounds or so.

http://cgi.ebay.com/ASUS-KFN32-D-SLI-SAS-MB-2-Opteron-Quad-Core-8347HE_W0QQitemZ300305454800QQcmdZViewItemQQptZLH_ DefaultDomain_0?hash=item300305454800&_trksid=p3286.c0.m14&_trkparms=72%3A1205%7C66%3A2%7C65%3A12%7C39%3A1%7C 240%3A1318%7C301%3A1%7C293%3A1%7C294%3A50

There are trustworthy sources on eBay offering new OEM Opteron 8346HE (1.8GHz) and 8347HE (1.9GHz). With careful watching, these can be had for $50US each. These quad cores are intended for 4-socket systems, but will work perfectly on 2-socket systems. So 8 cores for $100.

[Kayin]

Nice.

I get closer to a dually all the time.

[rubidium]

Here's (http://cgi.ebay.com/2-pcs-Opteron-8346-quad-core-1-8GHz-4-x-512K-L2-L3-2MB_W0QQitemZ330311722350QQcmdZViewItemQQptZLH_Def aultDomain_0?hash=item330311722350&_trksid=p3286.m20.l1116) a pair, brand new, for $114. I can vouch for this seller. They are 55W chips, so you'll hardly make any heat.

Edit: Also, here (http://cgi.ebay.com/AMD-Opteron-2354-Quad-core-2-2-GHz-L3-2MB-F_W0QQitemZ330320562341QQihZ014QQcategoryZ80142QQt cZphotoQQcmdZViewItemQQ_trksidZp1742.m153.l1262) is a single new OEM 2345 (2.2GHz) currently at $44, and 2 are available.

[Burnout21]

Here's (http://cgi.ebay.com/2-pcs-Opteron-8346-quad-core-1-8GHz-4-x-512K-L2-L3-2MB_W0QQitemZ330311722350QQcmdZViewItemQQptZLH_Def aultDomain_0?hash=item330311722350&_trksid=p3286.m20.l1116) a pair, brand new, for $114. I can vouch for this seller. They are 55W chips, so you'll hardly make any heat.

Edit: Also, here (http://cgi.ebay.com/AMD-Opteron-2354-Quad-core-2-2-GHz-L3-2MB-F_W0QQitemZ330320562341QQihZ014QQcategoryZ80142QQt cZphotoQQcmdZViewItemQQ_trksidZp1742.m153.l1262) is a single new OEM 2345 (2.2GHz) currently at $44, and 2 are available.

dude, thats crazy cheap! how much is a Quad socket board?

[ND40oz]

dude, thats crazy cheap! how much is a Quad socket board?

That's going to run you, plus you need a case to put it in, which can be a bit tough. Best way to go if you don't want to go all out modding a case is a barebones when you get into 4 and 8 way configs.

[Burnout21]

That's going to run you, plus you need a case to put it in, which can be a bit tough. Best way to go if you don't want to go all out modding a case is a barebones when you get into 4 and 8 way configs.

i can find a home for anything, just the average price of a Quad socket board plz, recomended boards that wont break the bank, no fancy features either.

[ND40oz]

Not too bad...

http://www.newegg.com/Product/Product.aspx?Item=N82E16813151089

Your power supply will need 2-4 pin and 2-8 pin connectors as well, but atleast it's EATX. And you'll need Server 2003 or 2008 Enterprise if you want to run windows.

[Burnout21]

Not too bad...

http://www.newegg.com/Product/Product.aspx?Item=N82E16813151089

Your power supply will need 2-4 pin and 2-8 pin connectors as well, but atleast it's EATX. And you'll need Server 2003 or 2008 Enterprise if you want to run windows.

So really for the average price of a Quad core I7 intel setup i could have a 16 core AMD setup, 4 times the amount of CPU's.

There seems to be more Intel 771 gear over here in the UK, which is annoying.

£500 could see me with a 16 core machine, that dose sound awesome. maybe i shall take two! render farm of 32 CPU's!

[rubidium]

So really for the average price of a Quad core I7 intel setup i could have a 16 core AMD setup, 4 times the amount of CPU's.

There seems to be more Intel 771 gear over here in the UK, which is annoying.

£500 could see me with a 16 core machine, that dose sound awesome. maybe i shall take two! render farm of 32 CPU's!

Sounds more and more like I'll soon be welcoming you to the growing 4x4 club!

[Burnout21]

Sounds more and more like I'll soon be welcoming you to the growing 4x4 club!

Bloody close mate, anyone got some benchmarks reviews of a 4x4 system against other plateforms?

If i do go 4x4, i shall most likely go straight onto a linux OS with my maxwell render software, i am off to the parents place this weekend to grab an old 939 system i have been storing there, so will try out the render farm software setup. And from there i shall work out if its worth it for me.

The 4x4 system would be done on the super cheap, most likely ran headless.

i have a spare 4GB (1GB modules) DDR2-800 ballistix so if the board dosen't require ECC memory at least i have a start point. To then add to.

I would rather miss out MS server OS's right now until Win7 is out, as there might be something new released?

[Snyxxx]

What are my options for Crossfire or SLI with a dual socket AMD or Intel server board? I have scoured the web pages and they seem to be very coy baout multi-GPU support.

Rubidium, I know yours supports SLI, any other Dual-processor choices with multi-GPU support?

[Karbon Killa]

I am right now working on a skulltrail rig for my first multi-cpu machine. So far I am loving it. I cannot wait to get my quad socket board. I want one SO bad.

[ND40oz]

What are my options for Crossfire or SLI with a dual socket AMD or Intel server board? I have scoured the web pages and they seem to be very coy baout multi-GPU support.

Rubidium, I know yours supports SLI, any other Dual-processor choices with multi-GPU support?

I think most of them will support SLI'd Quadros, but in terms of consumer cards, you're going to be stuck going crossfire on the intel side (except for Skulltrail). ATI doesn't seem to lock out any Intel chipsets that are capable of crossfire, I've run crossfired X1950XTXs on a 5000 series board and HD2900XTs in crossfire on a 5400 series board.

If you get an nVidia based board on the AMD side, it should support SLI.

[rubidium]

I think most of them will support SLI'd Quadros, but in terms of consumer cards, you're going to be stuck going crossfire on the intel side (except for Skulltrail). ATI doesn't seem to lock out any Intel chipsets that are capable of crossfire, I've run crossfired X1950XTXs on a 5000 series board and HD2900XTs in crossfire on a 5400 series board.

If you get an nVidia based board on the AMD side, it should support SLI.

I was about to say essentially the same things, but decided to eat breakfast first and got beaten to it.

A couple more thing I'll add:

Since multi-CPU socket boards generally appeal to the server/workstation audience and not to the gaming community, they tend not to advertise multi-GPU support, even though they may have it. You thus need to do more in-depth homework and download spec sheets or even user's manuals.

When you get into the spec sheets, look carefully at the system block diagram for the PCIe setup and the total number of signal lanes supported and how those lanes are allocated among physical slots. While there may be multiple x16 physical slots, they may not all be x16 signal lane slots, but rather when you load them up they are forced to share a limited number of signal lanes.

rubidium

[Snyxxx]

Thank you all for the replies. I guess I will need to dig deeper into the specifications/data sheets/manual to confirm a specific model.

[tomv1to]

£300 is the budget for mobo and CPU's so a dual Quad opty system i think is out of my budget range! lol!

Just looking for a cheap number cruncher

I don't want to sound like a negative nancy especially with this being my first post and all but wont a single i7 CPU beat a pair of Shanghai CPU's handsdown in most tests?

In which case you can pick up an MSI X58m motherboard for £125 and a single i7 920 for £205.

You'll probably go £50 over your £300 budget (after you've factored in a kit of DDR3,) but in the end you'll end up with a system that is easier to setup, OC and probably a lot more powerful (though I may be completely wrong.)

Does anyone know what the performance of a quad socket AMD system is like compared to an i7 is?