estimating network/Server needs

[si-skyline]

Hi everyone, This is a question to the network admins and network designers here...

I have a idea of launching a service and thats all it is at the moment a idea. I was wanting to know how do you know how meny people a server and internet connection handle?

If you said I need a system what can serve to 2000 people who can upload and download about 10MB of data as much as they like. how do you come to the conclusion that you need this internet connection and x y z server??

any ideas? or where to find more info?, do you know the actual term used for this kind of analysis?

Any comments would be a great help as I don't really know how to do it,

[SgtM]

Be prepared to spend upwards of $15,000/mo just for the internet connection. You're looking at getting a DS3. For the server.. Quad Core Xeon, at least 5Tb HDD space in a RAID5.

[si-skyline]

hi sgtm,

Being clear that 15,000/mo for the internet connection is per month right?
how doe you come to the conclusion of what I need? or is it just plain experience

What if I said I'm wanting to scale it down to around 200 people. How do I know the service what I'm going to provide is reliable?
I know to all computer systems there is a "tipping point" just not sure how to find it :/

Thanks once again for your comment +Rep!

[Luke122]

A quick estimate would be to calculate the maximum number of users to be online at one time, and how much speed/data they would each need. Build your system around the highest potential load it could see.

[si-skyline]

ah, thanks Luke. the easy solutions don't always shine thu xD

+REP!

[crazybillybob]

It's hard to say for sure with out knowing what kind of things you looking to do.
If your just hosting static pages, you can get away with a little 1U quad core.
If your looking at an e-commerce site with a product/order database your going to need a dual or quad CPU Quad core server with a goodly amount of space.


Plus if it's mission critical you'll need at least 2 servers setup in a cluster (active active cluster increases performance, but uses 2 licenses, and users may notice the slowness if one goes..) good rule of thumb never load a active active server over 65-70% at peek if you start getting to that point add another node or upgrade the nodes.
You maybe able to get away with some of the higher end Business class Cable internet connection (for the 200 user mark) but they have monthly max transfer limit, and once you break it they start charging you for the bandwidth, it could get expensive real fast.

This is one of what I liked to call the fun part of my job in the past, I had to design systems to handle things like what you want to do... Granted I had to know what software that was planned to be used , and at least a theoretical user load... then I added 30% just to be safe (call it future growth .... The suits love optimism :) )

I hope this helps...if you can give us more general info (while still being secretive) please do! it will help.

CBB
(yes the redneck can do computers)

[si-skyline]

Thank you a lot billybob :)

More information as you requested;

Core service of the business will be backing up. Type of services running in the beginning will be FTP later moving to RSync or something along those lines when I have a business program to handle the client side.
There is more services to go with that but won’t be implemented in the beginning.

I’m in talks with a certain university to support their users and have the potential to grow this business to 30,000 users + the teachers per rolling year from that source. This would be my main source of income at the beginning.

I want to expand into the business sector and also families to generate more customers.

I want the servers to manage the hosting of the service and also the space needed for the users. The website will prob be hosted on the likes of go daddy or something.

My idea for redundancy, scalability and reliability is to have two sites. For easy sake called "siteA" and "siteB"
siteA will be the main guy serving the users, this would have 1 / 2 servers in a cluster with an ISCSI box to give the storage information between the two.

siteB will be more of a lone wolf with a lot less to offer then siteA. This would be used for an offsite back up of the main server in siteA in case the whole thing goes up the wall.

If you or anyone else would like to give a hand because they think it sounds interesting or would just like to help don’t hesitate to PM me.

Hopefully this can be deployed by the end of next year to be up and running for the next academic year ready for the students who start university. The hardest part I’m facing at the moment knows what equipment will do the job and getting my point across to the suits to show this is a feasible idea and can greatly benefit some people


also a +REP!

[crazybillybob]

Ok,
You just changed the scale of the project :) If your focusing on a backup service for others then you have to be paranoid about data loss. What's that mean...
To start with your storage array should be some kind of S.A.N. defiantly configured to be fault tolerant Raid 50 (or Raid 5+0)... your going to need allot of space 15-30 Tb to start is not a crazy number. With that much space you need to take into account drive failure rates etc. as you can quickly lose allot of data if one disk fails. I'd recommend you get someone that does SAN work for a living to help you set it up, but it's hard to find people that are really good... There are allot of guys out there that say they know how to setup a S.A.N. Correctly but few that can actually do. (I know enough to be dangerous :) but that's why I've always hired people to do the mission critical jobs...) (ballpark for just one site's S.A.N. hardware $250,000 - $500,00 easy)

It sounds odd but you'll want multiple fiber interconnects to the storage (10Gb or Sun's infiniband fiber connections) and a good bit of memory. It's over kill but 2 4 way (4CPU) quad core Xeon (or Opteron) systems in a cluster setup is a good start.

I'd really look into Blade servers. They cost more on the front end but they do save on space (most data centers charge by the square foot) and power (both in usage and in power used for cooling Ie AC ). It would also give you more redundancy in the hardware, and with some of the High availability products out there from VMware and others your up time should stay in the high 99.98% or more.

Also remember you have to backup the data that is uploaded (this means tapes or optical ). the Ideal method for this is run the backup on your secondary site's san unit (make sure they setup syncing between the S.A.N. systems). That way he users don't notice any lag or latency when the backups run (which will take hours!!!) (ballpark cost for just the tape lib. $50,000)

Good luck,
I hope this helps
CBB

[si-skyline]

Thanks so much again :D

you have given me a lot of insight into what is needed for a larger scale network. Even tho I know what needs to be done its things like this what Im a bit fussy on.

I know in the beginning that I wont have a lot of users so that type of system might be a while off. In your personal opinion would be be better to rent from a Data-center instead of thinking to buy my own office space and then buy the servers and internet connection to run it?

To start and to know that the basic system will work what do you think using three personal computers in a cluster running on a 8mb ASDL home connection?

I know it sounds daft but that is what iv got at the moment that I can test on. do you think I could support about 20 users? of course I wont be charging for this at the moment but It would be used for BETA testing and other diagnostics. More of a proving ground that my setup is going to work. I think to impress people and make them understand a idea can work is having a working model of what you trying to do.

the current specs of my house is:
internet connection: BT ADSL 8mb down 512kb up

computer 1 : 3.00 GHZ P4
: 1 GB of Ram
: GIGA port
: 80 GB HD

Computer 2 + 3: 3.00GHZ P4
: 512 mb of ram
: GIGA port
: 500 gb HD

I could prob buy some upgrades such as ram and stuffs if it would make a big improvement.

Thanks once again for the help

+REP!

[crazybillybob]

For some back ground the last system I worked on had 2 sites, main site had 15TB of Storage 3 HPUX nodes, and 2 windows systems sharing the storage. it was for about 600 users, they loaded about 1Gb of Data into a Oracle DB each day and scanner 25000 pages of documents a day (each doc about 300Kb).
So I've had some experience with larger systems :) I also have done my far share of implementing cluster servers...Big car makers go crazy when there's unexpected down time :)

For a beta system...
I'd load a server class os on all my PCs', load it to the local HDD.

Load them all with 2Gb Ram (4Gb if they can take it)...

I'd also look into a cheap NAS you can get these for $150 (no disk) then throw 2 x 1Tb disks in it, config to Raid1. Use this like your SAN (because to properly cluster, all the servers need a disk that they can share (called a quorum disk)).

It's about a $300-$400 investment but well worth it to beta test... and worst case if this idea bombs you've got some storage for your home network :)

CBB

[si-skyline]

What I had in mind was to use the identical ones for the cluster, node1 + node2
then use the third to place the two 500gb disks in to make a 500gb raid1 config or the other option was to buy a third 500gb disk and start a raid 5 off or possibly a raid 6

node 3 of the cluster could be used as the data bank. with something like freeNAS or the same os as the others. do you think it would be better to get the NAS box instead?

is there a os you would recommend? at the moment im fairly good with opensuse, dunno how beowulf would size up for the clustering. i could of gone with the win2003 because i know how to use it. but then there is the fact of price for it :mad:

how do you feel about using a synology box?

thanks agian for your information, you are helping me so much

[crazybillybob]

Using the 3rd box as your NAS is fine. as most OS's don't like one of the nodes having different hardware. It throws off the load balancing and becomes the systems bottleneck. I'm not sure and software is better then the other for this, as there are so many ways to set something like this up. Each way has it's pluses and minuses... none are "perfect" but many do a good job at it. If you have some software that you've used or are comfortable with stick with it, as the learning curve on some of these is steep, and the beta is really just a proof of concept... quick and cheap to bring in backers/clients.

As for Os's I'm a Windows guy, I can make Windows server dance to any music that gets played. But there is the cost, one thing you could do is look into a MSDN subscription, I think the one that gives you all the windows OS's is around $3500 and it comes with a few free support calls (worth $300 each). Other wise you'll need the enterprise version of 2k3 or 2k8 to do clustering and it's not cheap....but you should be able to do volume licensing
that will cut down the cost (you'll need to buy the media, but it's only like $75 or so with a Volume lice. after your initial order you can add 5 client access lice at a time I want to say it's about $75 per seat...but not sure with the newer pricing from M$). You do not need to buy client lice for each server, just make sure the total client lice you have are more or equal to the max number of user you allow on the system (in this case system means both nodes together as one cluster) at one time.

I know I should learn more Linux/Unix and have been doing it but I'm not good enough to start designing systems that use more then the basic functions of it. (I still don't understand completely on how to get DNS and DHCP running and playing nice together on the same Linux box :) )


CBB

[si-skyline]

yeah im a windows guy too. the main and prob only reason of choosing a linux os is for the sake of cost. I know quite abit about linux, iv been learning on and off for about 3 years. thinking more im thinking of it a windows setup would be best, the thing that baffals me in the client licence. if im to buy licence for every user i support then it will be 75 for each one? it just makes it pointless if im charging a person 35 or something

or have i completely misunderstood that

[crazybillybob]

you need to have licenses for the max number of user that can be on the system at one time. So if you build the system to handle 35 people you'll need 35 licenses. Even if you have 500 paying users, the system can only handle 35 users at a time so you only need 35 lices.... make since???



(I'm out of town so it may take some time for me to reply... I'm checking don't worry I haven't forgot about you :) )


CBB

[si-skyline]

yeah, i tought it would be like that, thats no problem.
dont worry about me while your away. enjoy youself if you can xD

im too away at my mums so im doing more theory atm :)

iv also came up with a business name called

save our stuff, so its goes down to SOS :P