Archive for April, 2008 Page 2 of 2

Next Entries »

One Zope instance with two database

Published by
Jayarajan JN
at April 21, 2008 in Development. 0 

One zope instance two database files

a. Configuring zope.conf

We have to include multiple database files using multiple named <zodb> sections with one <filestorage> key in each which points to the corresponding Data.fs file.

<zodb 1>
    <filestorage>
        path $DATADIR/Data1.fs
    </filestorage>
</zodb>
<zodb 2>
    <filestorage>
        path $DATADIR/Data2.fs
    </filestorage>
</zodb>

We will be referring to the database using the corresponding ZODB label.

b. Connecting to the database programmatically

When Zope starts, it will automatically load the first database entry in zope.conf as default. So now we need to know how we can connect to the second database. The new connection is created using the existing connection.

    connection2 = connection1.get_connection('database_name')

But we need the existing connection to use get_connection(). It is easy if we have an object stored in current database

    siteManager = getSiteManager(some_object)
    defaultFolder = siteManager['default']
    # defaultfolder is an object in first database.
    conn2 = defaultFolder._p_jar.get_connection('2')
    conn2.root()['MyFolder'] = Folder()
    transaction.commit()    # myfolder references to an object in the other database
    myfolder=conn2.root()['MyFolder']
    siteManager.registerUtility(myfolder, IFolder)

c. Storing and retrieving the objects

Z3 referencing scheme is very advance to deal with for cross-database referencing. Once we have stored the object, all we need is a reference to access it. Which means, once we have a reference of the object, we don’t need to connect to database before trying to access the object. It will be totally managed by Z3. Retrieval of the object is the same as we do with any other object,

    # Use getUtility to grab the object faster
    folder_in_db2 = getUtility(IFolder)
    folder_in_db2['some_object'] = SomeObject()

You can assure that the object is stored in the second database by verifying the name of database.

    some_object._p_jar.db().database_name

Reference
zope api documentation on cross database references

Multiple database with Zope 3

Published by
Jayarajan JN
at April 21, 2008 in Development. 0 

When we deal with data intensive projects which are expected to be used by a large number of users, we should rethink about the architecture nine times. The best idea is to come up with a flexible, scalable model to build on. Performance, process and database load balancing, and fault-tolerance are the areas that deserve great attention while in the design phase. So how we can achieve such requirements in Zope 3?

It is known that Z3 does not provide any magic features to fully balance the load. However, there is one built-in method that allows us to have multiple identical Zope instances sharing the same database with help of ZEO servers. And for balancing the load among these instances we may use either Pound or Balance. Of course there are other alternatives too!

So what we have on hand to worry about is how to handle the database load. The key requirements that we have to meet are,

  1. High data availability
  2. Fault tolerance
  3. Data safety
  4. Scalability

In Z3, it gives us a few possibilities. And here we will discuss how we can use multiple database with Z3 for different use cases.

  1. One Zope instance with two database
  2. One Zope instance and ZEO server with two database
  3. One Zope instance with multiple ZEO servers
  4. Multiple ZEO servers sharing the same database

These basic setups can be scaled and combined together to design a setup to suit our requirement. All the best!

Next Entries »