Computer of the week... [Archive] - TBCS Community Forums

Airbozo

10-05-2006, 02:52 PM

Ok so this is something I have been thinking about for a while. I work with several different vendors systems from dell up to sun and sgi systems, desktop, rackmount, and server systems. I thought it would be informative to show you guys the insides of some of these systems so you can see how the "big boys" put thier systems together. Hopefully this will generate some ideas for people modding systems. Let me know if this is helpfull or if you have an questions. If I still have the systems here you can even ask for a specific angle shot.

First up is my current project (bulding 20 of these);

A SuperMicro dual Xeon 1u server. http://supermicro.com/products/system/1U/6015/SYS-6015B-T+V.cfm

http://www.lotechdesigns.com/host/images/2654supermicro_1u_dual-xeon-small.JPG

It has 2 dual core xeons, 16gb of memory and 500gb(2x250) sata disks.

Just at the bottom is a 700 watt slimline power supply. There is also a cooling shroud that covers the heatsinks and part of the memory to ensure proper airflow. This thing is NOISY due to the jet style fans... There is also a pcie riser card and an optional pcie 1x riser card. Notice on the top the two wierd slots, one is for an onboard raid controller and the other is for an IPMI card (Intelligent Platform Management Interface).

Mountain Dew

10-05-2006, 05:42 PM

i own a micro

they are nice servers
good luck

nil8

10-06-2006, 12:12 AM

Airbozo

10-06-2006, 11:03 AM

I've always found servers to be way more interesting to mod than regular machines. They can be engineered in a variety of ways and always astound me when I see them.

I wish more people would do server modding, but you're going to cover almost all of it up in a rack anyway, so it kinda makes sense.

Whatever company is paying you to do this has some good dough to be throwing down for 16gb of memory per machine.

Considering that they purchase ~10 systems like this a month I agree with the $$ statement. The next project for them is 15 dual opteron/xeon (not sure which yet) workstations with 12gb memory 1tb disk space and nvidia 7900 sli setup. They are inclined to go with the opteron due to the memeory handling and bandwidth but we are concerned about finding a reliable dual opteron motherboard that supports sli. We are still testing. I will post those systems when they come in.

nil8

10-06-2006, 02:19 PM

What kind of company needs dual xeons for workstations?

And how were you lucky enough to get your job?

progbuddy

10-06-2006, 02:27 PM

If I'm not mistaken, that looks exactly like a Apple G5 mobo...

Edit: omgosh, my new mobo has a IPMI for my HDDs

Airbozo

10-06-2006, 02:28 PM

What kind of company needs dual xeons for workstations?

And how were you lucky enough to get your job?

Not sure what they will be doing with the workstations just yet, they usually only buy rackmount systems, so I think it is something to do with the development of thier new product (online manuals and such). They are very tight lipped about this project just now. I do know they use some 3d design package just not sure which one.

Luck was not a factor in getting this job. They originally hired me for my SGI skills and then found out I can build _anything_. Just like every job I have had, there is good with the bad. The good: I get to work with some cutting edge gear. Bad: I also have to work on stuff that is really old... (mostly work on the older stuff, with new systems scattered in between).

OvRiDe

10-07-2006, 03:58 AM

I would have to say 3D is the first thing that comes to my mind. A buddy of mine works at a small graphic studio. They specialize in 3D rendering for Architecture firms. His is one of the "older" workstations that has dual 3Ghz Xeons, 8GB of RAM, and 400GB of 15K rpm SCSI drives. He said even that can get pretty bogged down when you are modeling objects with hundreds of thousands polygons in Lightwave. At night all the workstations run as render nodes so they can process their batch jobs while the machines would otherwise be idle.

Nagoshi

10-09-2006, 10:36 AM

If I'm not mistaken, that looks exactly like a Apple G5 mobo...

Edit: omgosh, my new mobo has a IPMI for my HDDs

If you didn'T knew already, Apple are now Intel-based computers.

Airbozo

10-09-2006, 11:30 AM

If I'm not mistaken, that looks exactly like a Apple G5 mobo...

Edit: omgosh, my new mobo has a IPMI for my HDDs

That is a SuperMicro mobo. I am pretty sure they use a proprietary mobo.

http://supermicro.com/products/motherboard/Xeon1333/5000P/X7DBR-i+.cfm

I like the description: Enhanced Extended ATX

heh

Stay tuned the next one will be posted today or tomorrow. I am snapping some shots of an older sgi workstation.

monoflap

10-09-2006, 12:44 PM

This might be a stupid question, but what kind of operating system would you run all that gear on. Because I know Windows has problems getting just 4Gb of ram to work.

ajmilton

10-09-2006, 01:07 PM

This might be a stupid question, but what kind of operating system would you run all that gear on. Because I know Windows has problems getting just 4Gb of ram to work.

the upper-end windows server 2003 products can apparently handle massive amounds of RAM. from what i saw on msft's website, server 2003 RC 2 SP 2 is supposed to be able to handle 2 TB of ram. i'm not sure what sort of configuration that's running though. it wouldn't surprise me if that was part of a cluster design.

of course, it wouldn't surprise me with that hefty a system that it's being used for database management or some sort of hardcore numbercrunching and running some *nix for that purpose.

but yeah, i doubt it's going to be running winXPPro :P

a.Bird

10-09-2006, 01:38 PM

How did you get a photo of my mobo? Did you sneak into my house or what??

Unfortunately, I'm having trouble running Duke3d on this thing. Any help??

Airbozo

10-09-2006, 02:02 PM

This might be a stupid question, but what kind of operating system would you run all that gear on. Because I know Windows has problems getting just 4Gb of ram to work.

Well they were going to run windows server 2003 x64 on these systems but found better performance under linux. I think they are running fedora core on these systems with a custom built kernel (stripped down except the necessary bits). Currently they are not doing very much clustering as each system services a specific customer or small group of customers. They also found that 16gb of memory is the "sweet spot" for thier applications. Meaning that more than 16gb does not provide much more performance for the $.

Airbozo

10-09-2006, 03:01 PM

OK Time for the next system. I may post 2 this week since this is an oldie.

This system is old (1993 era) by computer terms but I thought I would post it since there are a couple of interesting features that I like. Instead of posting pictures I am posting a link to a site that has all the pictures I was going to take, but much better quality.

The system in question is the SGI Indigo2 High Impact.

http://hardware.majix.org/computers/sgi.indigo2/purple.shtml

This system is cool because of the way it breaks down for service and some of the cooling features it has. The "feet" that transform the system into a tower model have built in cooling ducts that allow the airflow to be directed to the front. The drives are all sled mounted (sometimes the sleds are more expensive than the whole system), including the cdrom (you could also mount a third hd there). The side of the chassis opens to install graphics and option cards (yes that IS an EISA bus). This system also takes "normal" memory instead of proprietary memory like most sgi and sun workstations, and even though it takes 4 dimms at a time (for a total of 12), it does not matter if you install them from the front or back, the system will recognise and use what is installed. Originally the system was only capable of a maximum of 384mb, but with some of the newer dimms will actually work with 1gb. This is however unsupported by sgi, but then again the whole system is too old for support anyway. I personally have one of these systems and used to use it alot. Mine has the higher end graphics installed that take up 3 slots, plus a digital video option card. This chassis and case could be modded to hold a regular atx mobo.

Discuss....

nzcubey

10-14-2006, 10:44 PM

wow thats oldschool :D awesome

nil8

10-15-2006, 02:59 AM

That cpu heatsink is badass.
What do you use that thing for? Doom?

DaveW

10-15-2006, 03:49 AM

Philips made a graphics card? Wow!

...that thing could well be one of the first ever case mods. Did anyone think about that?

-Dave

bigbaddog

10-15-2006, 08:51 AM

Purple Computers :redface: ...reminds me of the surplus Intergraph workstations I used to buy/sell from Alabama DOT a few years ago. Old dual PII SCSI machines. Made a pretty good gaming tower back in 1999, except for the dual PCI graphic card...lots of memory for the day (80mb) but slow GPU.

Airbozo

10-15-2006, 01:12 PM

That cpu heatsink is badass.
What do you use that thing for? Doom?

Hehe I personally only use the one at work for testing/validating parts for customers. I used to use the one I own for some video editing until I wiped out the hard drive and lost the license key for the software. They would not renew it because I could not prove I owned the license (lost the receipt). It will run doom though. Doom 1.

DAVE: The graphics card was designed and spec'd by sgi, but alot of the chips were made by phillips and intel, then manufactured by sgi. That was the best video card ever made for the EISA bus...

These Indigo2's were direct competitors to the Integraph machines. At one point sgi was going to buy Integraph but instead only bought the rights to one of thier machines (zx10 I think).

simon275

10-16-2006, 05:28 AM

Philips made a graphics card? Wow!

...that thing could well be one of the first ever case mods. Did anyone think about that?

-Dave

Philips make alot of computer components not just tv's and such most of the memory used on nVidia cardss is made by Philips.

Any way how do you mange those big badass computers? Remotly through the network or do they have a vga out and you have one of those fold away screens in the racks.

Zephik

10-16-2006, 07:04 AM

That is pure awesomeness. I wonder what the price range on those are? lol I want one JUST to say I can say that I have one. They are... kind of pretty if you ask me!

-SF

Airbozo

10-16-2006, 11:05 AM

Philips make alot of computer components not just tv's and such most of the memory used on nVidia cardss is made by Philips.

Any way how do you mange those big badass computers? Remotly through the network or do they have a vga out and you have one of those fold away screens in the racks.

You can admin them through the network or serial port (all sgi's direct some output to either the console port or serial port #1). Of course since it is a graphics workstation most of the work is done on the graphics head. The monitor connector is a 13w3 (standard for a workstation of that era) _and_ a mini din connector for 3d glasses (some graphics cards have a sync connector too to sync the graphics from several different workstations). An interesting note: All sgi workstations use the console port (serial #1) to output errors to. If the system fails some diagnostic on power up (think post) then the error message is spit out on the console. If there is not enough good memory to boot the system it will give you an error on the console with the offending memory's location.

Airbozo

10-16-2006, 11:08 AM

That is pure awesomeness. I wonder what the price range on those are? lol I want one JUST to say I can say that I have one. They are... kind of pretty if you ask me!

-SF

I am pretty sure they can be had on ebay for a couple hundred dollars (originally sold for $10k +) They make good web servers, but the network port is only 10meg.

Zephik

10-16-2006, 05:05 PM

Airbozo

10-16-2006, 06:38 PM

$10K!!!? OMGOSH! That is absolutely insane. Well if thats case, I'll take three!:banana:

What were they originally used for again? lol Please dont tell me they were super calculators or something? :p

-SF

The indigo2's were originally used for graphics design and basically anything graphics related. HEAVY graphics issues. At the time they came out (1992?) there was nothing on the market that could touch them. GE medical still have a few running MRI systems (which we supply parts for). The parts are usually more valuble than the system as a whole...

Zephik

10-17-2006, 12:57 AM

Found one I think??

http://cgi.ebay.com/SGI-Indigo-2-Indigo2-R4000-GR3-XZ-Graphics_W0QQitemZ280038585904QQihZ018QQcategoryZ1 1223QQssPageNameZWDVWQQrdZ1QQcmdZViewItem

Airbozo

10-17-2006, 11:44 AM

Found one I think??

http://cgi.ebay.com/SGI-Indigo-2-Indigo2-R4000-GR3-XZ-Graphics_W0QQitemZ280038585904QQihZ018QQcategoryZ1 1223QQssPageNameZWDVWQQrdZ1QQcmdZViewItem

That is the lower end version of the purple one I showed. The purple indigo2 is called the impact and has a different power supply (more power) and eisa backplane to handle the high end graphics and video editing boards. BTW sgi systems are almost useless without the OS disks (talk to me if you need some...). Some systems will run a linux kernel, but the graphics won't work.

Airbozo

10-19-2006, 07:49 PM

Ok next up is a Super Micro S5035G-TB Workstation. This is not a particularly fancy or that interesting of a system but the reason I am posting it is to show the hot-swappable sata backplane (can be changed for scsi), and the fact that Super Micro put their boxes together neatly instead of just throwing things inside and closing it up. We get these system without proc, video card, or memory. This one is configured with a pentiumD 960 (3.6ghz), 2gb memory and a eVGA e-GeForce 7950 GT KO video card, 500gb sata drive and a dual layer dvd r/w drive. The customer will be running windows 2000 on it and upgrading to XPsp2 next year (yes they are behind the upgrade cycle).

Inside shot:
http://www.lotechdesigns.com/host/images/9027sm5035g-tbsmall.JPG

Sata backplane:
http://www.lotechdesigns.com/host/images/6137sm5035g-tb-satasmall.JPG

Drive sled:
http://www.lotechdesigns.com/host/images/6186sm5035g-tb-satasledsmall.JPG

We like the Supermicro systems since they just don't break. In the 3 years I have been here we have only had one failure in the field (out of a couple of hundred shipped). This workstation is super quiet (unlike their 1u systems which should come with a hearing damage warning label). I can barely hear it running and opened it up to double check that the fans were all turning...

Discuss...

nil8

10-19-2006, 09:25 PM

I've had numerous arguments with my coworkers about scsi vs sata and it's nice to finally see it fulfill the same needs as scsi.

Looks like it has good airflow and would work great for a standalone server. How many U is the case? Looks about 3 or 4 from my estimates.

Does the front bottom hold another drive sled or is that space reserved for intake?

Airbozo

10-20-2006, 11:24 AM

That system is a tower system, so I think it is 4u. SM makes a similar model to rackmount. The lower portion is for intake, but looking at it, it should be easy to throw some kind of drive mounting assembly in there.

Working with both sata and scsi, I prefer scsi for speed, but sata for size and noise. When recomending storage to customers, it all depends on the application they will be using. If they need high speed in a single system scsi all the way, if they don't mind a storage expansion cabinet, sata on the low end fibre channel on the high end. Equalogic makes a really impressive sata san box that is very close to or equal scsi speed. Most of our high end video editing customers and simulation systems use scsi due to the reliability and speed.

ajmilton

10-20-2006, 11:35 AM

supermicro's good stuff. we used their 1u servers for computational machines. slap a couple of processors, a couple gig of memory, and a hd into a rackful of 1u's, network 'em up and you have a beautiful rack of math pwnage :P

but yeah, they're really reliable. and the few problems we've had were handled very adequately.

i think the 1u's came with a prefab duct for the processors, which also made things nice.

Airbozo

10-20-2006, 02:13 PM

So out of 3 systems so far only one works. I personally think it is related to the heatsink. It takes _alot_ of pressure to push down on those clips to get them to secure to the board and the mobo flexes and touches the case. I have never liked those heatsinks for that reason. They also have a nasty habit of coming loose in shipment. The engineer that came up with that design should be shot. I have swapped out everything including the bios chip to try and fix it, but 2 in a row bad? Must be related to something I am doing (although in 20+ years of working on computers I have only once ruined a mobo by installing a heatsink, now maybe 3 times, twice in one day!?!?!?...)

Despite one bad tech support guy I talked to (I had to ask him if he was the receptionist), SuperMicro has been great. They are rma'ing the boards and we should be able to pick them up today (we are only a couple of miles from thier warehouse).

...And the sales guy just came in and we are taking one of the systems in to their QA guy to have him do some testing for us (really helps when you buy $100k+ of parts from them a month). I will post the results.

EDIT: Just got back from Super Micro and they were very helpfull. They put the system on their test bench and within less than minute they were telling me the memory was bad. Sure enough we swapped it out (I already swapped it out with known good memory) and the system came right up. I came back to my lab and grabbed all the good sticks of memory I had and went back to test it in the system. Some worked some didn't, _and_ the known good ones worked the first time we booted it but not the second time. Reseated everything and it worked. Tried the rest of the known good memory and some failed and some didn't (different ones this time). We came to the conclusion that we had some flakey memory and are having the vendor replace all of it with hopefully a different maufacturer. We are using ATP memory since it is a local company and they worked with Super Micro the get it certified in their systems. We are going to test some more of the ATP memory and if it fails or is flaky again we are not buying from them ever again.

update: sales guy just came in and we are going to take this memory over to their lab and verify that it is bad or good on a memory tester.

markkleb

10-23-2006, 05:46 PM

I really like how clean the insides are. Thanks for the post.

Who makes the Slimline PS?

Airbozo

10-23-2006, 07:51 PM

I really like how clean the insides are. Thanks for the post.

Who makes the Slimline PS?

Yes SM does a good clean assembly. Makes my job easier since I just have to add parts and not clean up the whole unit.

You can find slimline psu's on newegg and other places. Try searching for 1u power supplys or even 2u. The 1u's can sometimes be noisy. Hell one of the systems here has a 700w 1u power supply that sounds like a small jet on it's quietest setting. Not sure why it needs a 700w ps though since there is only room for 4 drives and no pcx-e16 slot... Only a dual xeon too.

BTW Any requests? I was planning on showing the insides of an sgi supercomputer or a couple of sun boxes.

Maz

10-24-2006, 06:08 PM

i like the sound of "super" better than "box" so please show the super : )

nil8

10-24-2006, 10:28 PM

Yea, supercomputer gets my vote.
You have a cray sitting around? ;)

Airbozo

10-25-2006, 11:33 AM

Yea, supercomputer gets my vote.
You have a cray sitting around? ;)

Hehe not anymore. Used to have a cray in one of my old labs. was crazy fun to watch since it had a liquid cooled backplane... http://vitalsecurity.typepad.com/photos/computer_history_musuem/pic_0077.jpg

I will line up an sgi supercomputer system and snap some pics.

Airbozo

11-01-2006, 03:20 PM

Got busy this week so I will try and get some pictures up tomorrow or friday.

nil8

11-01-2006, 03:39 PM

Imagine the amount of work the @home folding project could do with one of those crays...sweet lord.

Airbozo

11-01-2006, 04:04 PM

Imagine the amount of work the @home folding project could do with one of those crays...sweet lord.

I am pretty sure that there is a couple of crays working on that project but probably not the newer ones. I know that there are some sgi supercomputers involved in seti and folding projects.

Omega

11-01-2006, 09:46 PM

This might be a stupid question, but what kind of operating system would you run all that gear on. Because I know Windows has problems getting just 4Gb of ram to work.

I believe the 4gb cap is for WinXP 32bit edition. I know that windows XP 64bit can support either 8bg of 32, i forgot.

nil8

11-01-2006, 11:10 PM

I don't think they would run windows xp on most of these machine. It would probably be some flavor of linux or windows server.

Airbozo

11-05-2006, 05:07 AM

I don't think they would run windows xp on most of these machine. It would probably be some flavor of linux or windows server.

That is correct sir! They are runing a custom smp linux kernel optimized for io and network ooperations and nothing more. They won't even let us do the install for them since they have some propritary stuff in their kernel.

Sorry I have not updated this last week. I have been swamped at work. In one project I had to get this: http://www.wacom.com/lcdtablets/index_21UX.cfm working in windows 2003 x64.

We are being audited on monday on our install procedures for windows 2003 server x64 on dell 690's. Stupid government contractors...

After wednesday I will get some more systems up.

fragged

11-10-2006, 04:32 AM

Airbozo

11-10-2006, 12:19 PM

wow,

is there anything wrong with having a sexual attraction to servers? I love them, I'm planning to do a 99.9% uptime for a year project with two servers (one will be a backup server) so i can be a poonage seeder on bittorrent when i get adsl2+ (12mbit down, 1mbit up, only download counts towards cap)

Hehe isn't that what makes us geeks?

My O2 at home has a 99.9% uptime for the last year and a half. Lost power twice in the last year and that is the only time it has gone offline. (hey I live in the mountains so twice in one year is is an improvement!)

nil8

11-10-2006, 03:12 PM

Don't you have your systems on UPS?

Those lcd tablets are freaking sleek.

Airbozo

11-10-2006, 08:27 PM

Don't you have your systems on UPS?

Those lcd tablets are freaking sleek.

Yes both of my important systems at home are on a ups, but the power went out for 3 days. I only use my generator when the food in the freezer starts melting...

I am still drooling over that lcd tablet. When I saw it come in for testing I thought "Oh, nice, but I bet the display sucks". I was wrong. that display looks as good as _any_ 21" flat panel I have ever seen. I just wish I had enough time to load a couple of games on the system to see how it would work in say doom. Touch the baddie with the pen and your guns blaze perfectly on target.

I was able to test it with photoshop and it was amazing the control over the details I had. The company that gets the tablet is testing it for a new product they will be introducing. They sell a product called "CyberKnife" Which is a robotic radiosurgury operating system used for removing tumors, growths or any intricate surgury that can be automated. They link the mri into the software, design the operating procedure, test it extensively then operate on the paitent. The tablet will be used to manipulate the mri images and to design the operation.

http://www.accuray.com/

nil8

11-11-2006, 03:28 AM

Ok, 3 days I understand. The way you made it sound it was a few minutes.

That better be a nice display for its pricetag.

Medical technology is a rapidly growing field and rather sweet to see.
I've worked for a couple of hospitals and helped install 36" touchscreens in the surgery rooms for documentation. It was rather impressive. Except suiting up and having to schedule around all the surgeries.

Airbozo

12-01-2006, 07:18 PM

Sorry for the lack of updates here. I have been _really_ busy at work and hope to have another system up early next week. Takes time to tear some of these systems apart and put them back together.

Please Stand By...

AJ@PR

12-01-2006, 07:43 PM

Please Stand By...
Definitely.

I hadn't written anything here... but, man... :D

This thread is great!
Very cool hardware.

Hey, can anyone upload pics of server rooms?
All I have is the CDW advertisements.... where the dude is standing in front of a wall of rackmounts. Server-room pics make me drool.

I'm sorry, either upload pics, or (easier), provide a link. :)

Airbozo,
Thanks for the updates.
Rock on!!!

AJ

nil8

12-01-2006, 09:23 PM

Find a server farm and see if they've give you a tour. Some will, some won't. Most geeks understand the drool factor when it comes to server rooms. It's really impressive. I worked in a hospital with one that was sweet. Over 90 servers.

Whenever you get a chance Airbozo, we understand. Holidays and all. Congrats on your wife too.

AKA_RA

12-01-2006, 09:39 PM

i believe i started salivating at the simple mention of a server farm. lol.

Airbozo

12-14-2006, 11:42 PM

Ok, Sorry for the absence in this thread, work and home have been busy. Guess that is good at work so I still have a job and busy at home mean I am getting some things done and modding/improving my house.

I finally got a chance to take a few pictures of an sgi origin 300. (very similar to this 350: http://www.sgi.com/products/servers/origin/350/overview.html )
Only 73 gb scsi drive going into this system with 2 gb of memory, and 4x600mhz procs.
These systems are based on a mips chip like the older sgi systems. If I remember right this is the last model that has the mips chip. Sgi has dropped the mips chip in favor if the Intel Itanium. The Altix line from sgi features this chip running linux. I will get to that system in the next few weeks.

Any way on to this pics,

Here is a shot of the front of the system with the drives pulled out (one drive, one slot filler needed for proper airflow);
http://www.lotechdesigns.com/host/images/7330DSC01518.JPG
The display is part of the L1 controller that allows you to do some pretty complex diagnostics and information retrieval over a serial port without actually powering up the system. It also allows you to power up the system and can also act as the console. The L1 Controller i running a mini version of linux on a chip.

This is the back of the system. You can see the I/O card on the right. this is what makes this system a "Master" unit. Without this card, all you can do is hook it up to another master Via the ports on the far left, called a numalink (they may have changed that name). The I/O card has 2 serial ports, a console port (also allows you to access the L1), trigger ports (between the console and serial), 2 usb ports, a network port and that port to the left of the network jack is a direct connect to the L1.
http://www.lotechdesigns.com/host/images/7705DSC01520.JPG

Here is a shot of the innards. Memory on the top left, bottom left is a top view of the I/O card, The 4 smaller black heatsinks are the processors, the one between the 2 procs is the memory interface controller (plus some other stuff), the heatsink on the far laft is the crossbow chip. This is what allows the system to communicate across the 2 numalinks so fast. You can see several power regulators also on the board. The power supply is on the bottom right.
http://www.lotechdesigns.com/host/images/25DSC01516.JPG

This is a shot of the memory module. I should have put a regular ddr2 next to it so you can see how big it is. It is only 512mb ecc (dual ecc or something like that), buffered.
http://www.lotechdesigns.com/host/images/7657DSC01519.JPG

Let me know if you want a different shot.

AJ@PR

12-15-2006, 12:04 AM

If it isn't too much to ask, more pics! :)

I find it interesting that 'for proper airflow' you need to instal drives/slot fillers.

That L1 Controller thing also sounds interesting.

What would the benefit be of running one Master with various non-Master (slaves?) ones hooked up to it?
Better performance?

The amazing-ness of sgi...

Airbozo

12-15-2006, 02:46 AM

I will get some more shots tomorrow...

The slot filler just keeps the air circulating around the drive bay instead of just rushing through the empty slot (path of least resistance).

The L1 is fun to play with and you can hook up 16 (or more) of the nodes to act as one system and all the L1's are then controlled by an L2 (a larger computer running an embedded version of linux) across the numalink. And communicate via a router, that controls the data flow on the numalink.

The operating system as well as all your applications would see _one_ system with 64 processors and the total sum of all the memory in all nodes (I think the max is 4 gb per node x16). The hardware along with software hooks, make sure that the data that the local cpu is working with stays in local memory and not in memory several links away, although the local cpu would have access to data in remote nodes.

couple of things I forgot to mention; You can have several "slaves" that also have I/O panels in them and that I/O panel would be usable by any node. Also each L1 would have it's own "address", usualy defined by the rack number and position in that rack. For example, an L1 address of 001c14, would indicate rack 001, position 14 (counted in 1u slots from the bottom of the rack). The c indicates it is a compute node, an r would indicate a router node and a p would indicate a pci brick, with 4 busses and 12 slots, g graphics node (there are a few more).

Airbozo

03-12-2007, 10:25 PM

Whew! long time since new systems. Shame on me!

These are some pictures of the inside of a dell inspiron 690. A customer of ours uses these for workstations used by a Dr. to plan laser surgery. Pretty cool stuff, but I have never seen the software that runs on it. I am hoping to see a demo soon. They use the Wacom Cintiq display for the interface and display.

http://www.wacom.com/cintiq/index.cfm

Here is a shot of the inside. Top left, external drive bays, 2x5 1/4, 2x3 1/2. Top right, power supply. Its a big beast and takes a huge cord. Just below the power supply are the 4 drive bays. SAS drives or SATA. Both controllers are on the mobo, just need to switch the cables. Middle is the video card. Also a beast. Bottom left, processors, bottom right, memory;
http://www.lotechdesigns.com/host/images/631DSCF0072.JPG

Here is a closer shot of the video card. It's an nVidia quadro 4550. Notice the board extender and metal bracket to give it support and make it long enough to be supported by the end slot. Also notice the middle blue plastic support bracket;
http://www.lotechdesigns.com/host/images/8594DSCF0073.JPG

Here is a shot of the airflow plastics removed from the processor and memory. Huge fan on the left and another on the right(a 120mm);
http://www.lotechdesigns.com/host/images/8309DSCF0074.JPG

Here are the memory riser's. The plug directly into the onboard memory slots and provide for quad channel memory access. There are 2 of them in each machine with 16 slots each. We only configure the system with 8 gb of memory;
http://www.lotechdesigns.com/host/images/9062DSCF0075.JPG

Here is a closer shot. Notice the heat spreaders on the memory and how the whole assembly plugs into 2 memory slots;
http://www.lotechdesigns.com/host/images/5353DSCF0076.JPG

That's all for now. Hopefully it won't be 3 months before the next post.

Zephik

03-13-2007, 02:26 AM

You have the best job in the world. nuf said.

lol

Spawn-Inc

03-13-2007, 01:52 PM

Here are the memory riser's. The plug directly into the onboard memory slots and provide for quad channel memory access. There are 2 of them in each machine with 16 slots each. We only configure the system with 8 gb of memory;
http://www.lotechdesigns.com/host/images/9062DSCF0075.JPG

Here is a closer shot. Notice the heat spreaders on the memory and how the whole assembly plugs into 2 memory slots;
http://www.lotechdesigns.com/host/images/5353DSCF0076.JPG

That's all for now. Hopefully it won't be 3 months before the next post.

can this be put in any old system?!? and is there an ddr one available too? also does anyone know what would happen if i made a board like that but instead of having all those resistors and caps if i just connected it straight to the ram only. i mean like this

the blue lines are suspose to represent the 240 pins all being connected. would the computer work or think it has (lets say you put a 1gb stick in each slot) 8gbs in the 1 slot?
http://inlinethumb45.webshots.com/3564/2386881220100511463S600x600Q85.jpg

Airbozo

03-14-2007, 11:19 AM

After checking with some friends, I would have to say that not just any riser card would work. It must contain some filtering circuitry and some logic to deal with timing issues. I just tried to use a SUN memory riser in a generic server board I have here and just got the memory error beeps. Checked everything out and I did not burn anything up in the process since the system still boots and see all the memory.

Airbozo

03-14-2007, 07:36 PM

Yes the 690's take 2 people to lift out of the box (or at least one Governator). They are built tough too. Big honking metal and little plastic.

RE: Power; Yes Jeremy you are right about the power issue. I forgot to point out the power wires coming off of the riser boards. They need one connection per riser (there are 2 in the picture).

I did a quick search on Google and found a couple of different memory riser cards. Apple (for the MAC Pro), Dell, HP, IBM, all sell riser cards for some of the systems they produce. I would surmise that all of them are different but would need to verify that by testing them.

On a different note, I am getting ready to put together a couple different clusters. One for a web server, one for a visualization cluster (2+ graphics nodes with one master controlling the output of each node). If there is enough interest I will post the step by step procedure with screen shots of the install. (at least for one of them, then annotate the differences in the two) I will be using the Rocks Clusters software.

http://www.rocksclusters.org

This software is based on CentOS which is based on redhat (not sure of the version), but you can actually use ANY distro you want.