Author Archive for Jayarajan JN

Multiple ZEO servers sharing the same database

Multiple Zope instances

As shown above, the most common way to load balance for Zope setups is to create multiple identical Zope instances sharing the same database via the ZEO.

Multiple ZEO servers sharing the same database

The less common way, shown above, is to create redundant ZEO servers which shares the same database. So, we must make sure that failure in one ZEO server won’t stop our Zope application from running correctly. The solution is to keep multiple copies of the Data.fs synchronized and available under separate ZEO servers. But syncing can be expensive in the case of bulky database. In any case, Zope allows multiple server addresses where the Zope instance can look for a particular ZEO server to get a particular database. The configuration is to be done in zope.conf:

    <zodb 1>
        <zeoclient>
            server 192.168.0.1:8800
            server 192.168.0.2:8800
            server 192.168.0.3:8800
            storage 1
            # ZEO client cache, in bytes
            cache-size 20MB
            # Uncomment to have a persistent disk cache
            #client zeo1
        </zeoclient>
    </zodb>

In this case, the Zope instance will sequentially iterate among the server addresses to find a ZEO server to access database ‘1′.

One Zope instance with multiple ZEO servers

One Zope instance with multiple ZEO servers

In the previous setup, we had multiple database taken care by a single ZEO server. For this setup, we can have separate ZEO servers to take care of each database file. For that we will need to create another ZEO instance with a different socket-connection address (ip-address:port) And we may then reconfigure the zeo.conf files and zope.conf as follows:

Let us say first ZEO server caters the 1st database and the second ZEO server deals with the 2nd database.

a. Configure the first zeo.conf

The difference is in the file storage section. We need a filestorage section to include the first database. The rest of the configurations are default.

    <zeo>
        address 8800
        # when we run the zeo servers in different machine,
        #its good practice to keep the port id same for all zeo servers
        read-only false
        invalidation-queue-size 100
        # pid-filename $INSTANCE/var/ZEO.pid
        # monitor-address PORT
        # transaction-timeout SECONDS
    </zeo>
    <filestorage 1>
        path $INSTANCE/var/Data1.fs
    </filestorage>

b. Configure the second zeo.conf

As we said this ZEO server will handle the 2nd database file.

    <zeo>
        address 8900
        # when we run the zeo servers in different machine,
        #its good practice to keep the port id same for all zeo servers
        read-only false
        invalidation-queue-size 100
        # pid-filename $INSTANCE/var/ZEO.pid
        # monitor-address PORT
        # transaction-timeout SECONDS
    </zeo>
    <filestorage 2>
        path $INSTANCE/var/Data2.fs
    </filestorage 2>

c. Configure the zope.conf

So now all we need to do is make a few changes to the zope.conf to get access to the database via the ZEO servers.

    <zodb 1>
        <zeoclient>
            server localhost:8800
            storage 1
            # ZEO client cache, in bytes
            cache-size 20MB
            # Uncomment to have a persistent disk cache
            #client zeo1
        </zeoclient>
    </zodb>
    <zodb 2>
        <zeoclient>
            server localhost:8900
            storage 2
            # ZEO client cache, in bytes
            cache-size 20MB
            # Uncomment to have a persistent disk cache
            #client zeo2
        </zeoclient>
    </zodb>

And we don’t have to make any change to the code to change to this setup too.

The real advantage of zeo can be experienced when we use it with multiple zope instances.

Later we will see how to setup multiple zope instances sharing multiple database.

One Zope instance and ZEO server with two database

One Zope instance and ZEO server with two database

This setup will introduce the ZEO server using mkzeoinstance and we will have to modify the configuration files, specifically zeo.conf and zope.conf. That’s it! No need to change the code. Connecting and dealing with objects is all the same.

ZEO server is the native solution for database load balancing. it allows us to access the Data.fs over the network. Also, it takes care of object locking!

a. Configuring zeo.conf

In zeo.conf, what is important is to specify the database files it should manage.

    <zeo>
        address 8800
        read-only false
        invalidation-queue-size 100
        # pid-filename $INSTANCE/var/ZEO.pid
        # monitor-address PORT
        # transaction-timeout SECONDS
    </zeo>
    <filestorage 1>
        path $INSTANCE/var/Data1.fs
    </filestorage>
    <filestorage 2>
        path $INSTANCE/var/Data2.fs
    </filestorage>

So now we have our database stored at the ZEO server.

b. Configuring zope.conf

Now we must configure zope.conf to get access to the database via the ZEO server.

    <zodb 1>
        <zeoclient>
        server localhost:8800
            storage 1
            # ZEO client cache, in bytes
            cache-size 20MB
            # Uncomment to have a persistent disk cache
            #client zeo1
        </zeoclient>
    </zodb>
    <zodb 2>
        <zeoclient>
            server localhost:8800
            storage 2
            # ZEO client cache, in bytes
            cache-size 20MB
            # Uncomment to have a persistent disk cache
            #client zeo2
        </zeoclient>
    </zodb>

When the ZEO server and Zope instance are started, the Zope instance will look for the ZEO server and it will connect to the database. The Zope instance will keep on searching at a regular interval if it fails to find the ZEO.

We don’t have to make any changes in the code to migrate from the previous setup to this setup.