How do the large scale sites do it?

[Jim Jones]

I am looking to set up a fail safe site. My idea is to setup three
_duplicate_ sites with three different webhosts. The DNS provider
(Zoneedit) provides a distributed failover system; the DNS host tests the
primary website, and if it fails to respond, the DNS record is altered to
point to an alternate site.

The problem that I face is keeping the databases on the three different
hosts in sync. I'm not talking about a master/slave replication; I need
the data to be consistent across all locations at all times without any
conflicts (duplicate keys, etc). Replication doesn't help me because what
could happen is the master system goes down, users are redirected to a
secondary host, the user makes a change on a secondary system, the primary
system comes back up yet it doesn't know about the changes that took place
on the secondary system.

Looking at the MySQL docs, I can setup a simple Master/Master system, but
this quickly breaks down when the structure of the tables are less than
simplistic (primary key conflicts, etc).

How do the big sites do it? How do they maintain data integrity across
multiple servers when any one of the servers at any time could take over
primary responsibilitie s?

Any help is greatly appreciated.

[codeWarrior]

Re: How do the large scale sites do it?

They host the database on a separate database server.... Look at
www.Pair.com for example....

Their web sites are on about 200 different servers... Their mySQL databases
are on about 45 data servers...

webhost1 --> uses dataserver1
webhost2 --> uses dataserver1
webhost3 --> uses dataserver1
dataserver1



"Jim Jones" <webmaster@mark etnoize.com> wrote in message
news:9xvOa.1388 $4n6.120305@new s.uswest.net...[color=blue]
> I am looking to set up a fail safe site. My idea is to setup three
> _duplicate_ sites with three different webhosts. The DNS provider
> (Zoneedit) provides a distributed failover system; the DNS host tests the
> primary website, and if it fails to respond, the DNS record is altered to
> point to an alternate site.
>
> The problem that I face is keeping the databases on the three different
> hosts in sync. I'm not talking about a master/slave replication; I need
> the data to be consistent across all locations at all times without any
> conflicts (duplicate keys, etc). Replication doesn't help me because[/color]
what[color=blue]
> could happen is the master system goes down, users are redirected to a
> secondary host, the user makes a change on a secondary system, the primary
> system comes back up yet it doesn't know about the changes that took place
> on the secondary system.
>
> Looking at the MySQL docs, I can setup a simple Master/Master system, but
> this quickly breaks down when the structure of the tables are less than
> simplistic (primary key conflicts, etc).
>
> How do the big sites do it? How do they maintain data integrity across
> multiple servers when any one of the servers at any time could take over
> primary responsibilitie s?
>
> Any help is greatly appreciated.
>
>[/color]

[Jim Jones]

Re: How do the large scale sites do it?

> Their web sites are on about 200 different servers... Their mySQL
databases[color=blue]
> are on about 45 data servers...[/color]
I'm a bit confused; you state "45 data servers" yet your diagram just
specifies one data server.

If there really are 45 data servers, then I'm back to my original question.
Other than a simplistic website, what do they do when there is a high
interdependence of data across multiple servers?

e.g. If there are 45 servers, each with a table with an autoincrement field,
in order for one row to be inserted into the table on server X, server X
must first verify that servers A,B,C haven't already utilized that key. So
how does this synchronization take place? Or do they just have one massive
MySQL server, and assume the risk of a single point of failure?


"codeWarrio r" <GPatnude@HotMa il.com> wrote in message
news:cGDOa.203$ 5o5.197888@news 1.news.adelphia .net...[color=blue]
> They host the database on a separate database server.... Look at
> www.Pair.com for example....
>
> Their web sites are on about 200 different servers... Their mySQL[/color]
databases[color=blue]
> are on about 45 data servers...
>
> webhost1 --> uses dataserver1
> webhost2 --> uses dataserver1
> webhost3 --> uses dataserver1
> dataserver1
>
>
>
> "Jim Jones" <webmaster@mark etnoize.com> wrote in message
> news:9xvOa.1388 $4n6.120305@new s.uswest.net...[color=green]
> > I am looking to set up a fail safe site. My idea is to setup three
> > _duplicate_ sites with three different webhosts. The DNS provider
> > (Zoneedit) provides a distributed failover system; the DNS host tests[/color][/color]
the[color=blue][color=green]
> > primary website, and if it fails to respond, the DNS record is altered[/color][/color]
to[color=blue][color=green]
> > point to an alternate site.
> >
> > The problem that I face is keeping the databases on the three different
> > hosts in sync. I'm not talking about a master/slave replication; I[/color][/color]
need[color=blue][color=green]
> > the data to be consistent across all locations at all times without any
> > conflicts (duplicate keys, etc). Replication doesn't help me because[/color]
> what[color=green]
> > could happen is the master system goes down, users are redirected to a
> > secondary host, the user makes a change on a secondary system, the[/color][/color]
primary[color=blue][color=green]
> > system comes back up yet it doesn't know about the changes that took[/color][/color]
place[color=blue][color=green]
> > on the secondary system.
> >
> > Looking at the MySQL docs, I can setup a simple Master/Master system,[/color][/color]
but[color=blue][color=green]
> > this quickly breaks down when the structure of the tables are less than
> > simplistic (primary key conflicts, etc).
> >
> > How do the big sites do it? How do they maintain data integrity across
> > multiple servers when any one of the servers at any time could take over
> > primary responsibilitie s?
> >
> > Any help is greatly appreciated.
> >
> >[/color]
>
>[/color]

[LeagueWebmaster]

Most larger firms use clustering to ensure their databases are always available. With MySQL there are DB nodes that then access the clustered servers. Personally I've not used the MYSQL clustering as most providers and services don't provide it as part of thier standard services. I imagine you would incurr a farely large amount of initial support trying to get two ISP to cordinate enough to push this through on MYSQL. Look here to get more information