Archive of UserLand's first discussion group, started October 5, 1998.

Re: A Limit on entries in tables?

Author:Dave Winer
Posted:9/3/1999; 3:33:56 PM
Topic:A Limit on entries in tables?
Msg #:10542 (In response to 5000)
Prev/Next:10541 / 10546

I've been thinking about this as the DG here passed 10,000 messages last week. The key is to have reasonably low-level access routines, so that when a system grows you can swap in a different storage method.

When I can clear a little time I'm going to give it a try here. I'll move all the messages to a hierarchic store, not a flat table, probably broken down by month. Then there will be a table that maps ranges of msgnums to months.

Then I'll empty out the messages table. The scripts will look in the messages table first, if it fails to find the message, it looks in the hierarchic store and copies the message into the messages table and adds its msgnum to a recently-used table which is sorted on date of last access. This process repeats until some fixed maximum number of entries are in the messages table, at which point, it swaps out the least recently used message back to the hierarchic store.

This is based on the theory that at any given time there are only about 100 or so messages that are actively being read. (We could test that assumption.)

Could the same method be applied to membership? Almost certainly yes.

There are responses to this message:

This page was archived on 6/13/2001; 4:52:24 PM.

© Copyright 1998-2001 UserLand Software, Inc.