Writing a program that uses persistently stored data (that is, data stored in an ObjectStore database) is similar in many respects to writing one that uses transiently stored data (that is, data that exists only while your program is running). When persistent data arrives in your program, it is already in the same format as the transient data. This means (for example) that you can use the same pointers and perform the same pointer operations with persistent data as you can with transient data. The decision to allocate persistent or transient storage is independent of the type of value you want to store; any type of value can be stored in either persistent or transient storage.
For more specialized topics (for example, persistent unions), see Chapter 1, Advanced Persistence, of the Advanced C++ API User Guide.
Establishing a Page Fault Handler
As explained in ObjectStore's Memory Mapping Architecture, ObjectStore uses page faulting to detect program references to persistent memory. To do this, ObjectStore must establish a fault handler to detect page faults at the top of every program stack in your application.
To establish the fault handler, add the following macros to your application:
int main(int argc, char* argv[])
{
OS_ESTABLISH_FAULT_HANDLER {
// your code
} OS_END_FAULT_HANDLER
}
The use of braces to enclose the code is not required, but some text editors might object if you do not use them. If your application uses threads, you also must insert the macros at the beginning and end of any thread that performs ObjectStore operations.
For detailed reference information about the fault-handler macros, see the following in the C++ API Reference:
For detailed descriptions of ObjectStore's overrides of both operators, see ::operator new() and ::operator delete() in the C++ API Reference.
::operator new()
ObjectStore overrides ::operator new() for use when allocating persistent storage. In addition, ObjectStore provides overloadings of ::operator new() to specify where to allocate the storage and the type of object to be allocated. These overloadings use the placement syntax of the C++ ::operator new(). Syntax
The following is the generalized syntax for the overloadings of ::operator new():
void* ::operator new ( object_size, where, typespec [ , how_many ] )
where can be a pointer to an os_database, os_segment, or os_cluster object. This argument specifies the database, segment, or cluster (respectively), where you want ObjectStore to allocate. For more information, see How ObjectStore Allocates Persistent Storage. where can also be an os_cluster_with object; see Clustering.
To allocate storage in the same database, segment, or cluster as a previously allocated object, you can use os_database::of(obj), os_segment::of(obj), or os_cluster::of(obj) as the where argument. All three functions take a pointer to the stored object (obj) as the argument and return a pointer to the database, segment, or cluster in which obj was stored. os_cluster::with() can also be used with new to supply a pointer to a cluster, but it has somewhat different semantics; see Clustering.
typespec is a pointer to an os_typespec object that ObjectStore uses to determine the type and layout of the object. For more information about this argument, see Using Typespecs.
how_many is used only when allocating an array. It is a signed 32-bit integer of type os_int32 that specifies the number of elements in the array.
Examples
In its simplest form, ::operator new() requires only two arguments, where and typespec, as in the following example. The example assumes that the arguments this_cluster and obj_ts are defined elsewhere in the program.
obj_type *obj = new(this_cluster, obj_ts) obj_type;obj will be stored in the cluster to which this_cluster points. obj_ts is the typespec for obj.
In the next example, ::operator new() creates an array of 10 elements of obj_type objects. Note that this overloading uses the same typespec argument as the previous example used; there is no os_typespec specifically for arrays. The how_many argument (10) identifies the number of elements in the array.
obj_type *some_objs = new(this_cluster, obj_ts, 10) obj_type[10];
How ObjectStore Allocates Persistent Storage
As explained in Persistent Storage: Organizational Overview, the cluster is the unit of allocation in a database. This means that, regardless of whether you specify a database, segment, or cluster for the where argument to ::operator new(), ObjectStore always allocates from a cluster. If you specify a pointer to a cluster as the where argument, ObjectStore allocates from that cluster. If you specify a pointer to a segment or database as the where argument, ObjectStore usually allocates from the default cluster.
Using new to Create Transient Objects
If your application requires both transient and persistent versions of an object, you can use ::operator new() to create either one.
Use one of the following methods to determine whether the database, segment, or cluster is transient or persistent:
To determine whether an object has been allocated persistent or transient storage, pass the address of the object to objectstore::is_persistent(). This method returns nonzero (true) if the address points to persistent storage and 0 (false) if it points to transient storage.
::operator delete()
ObjectStore overrides ::operator delete() to release either transient or persistent storage. ObjectStore makes the determination at run time, based on whether the object for deletion is transient or persistent. As with ::operator new(), before using ::operator delete() to release persistent storage, you must have created a database, opened it, and begun a transaction. For information on creating and opening databases, see Chapter 4, Basic Database Operations. For information about transactions, see Chapter 5, Transactions.
If you want to provide your own overloading of ::operator delete() to perform application-specific transient deallocation processing, you must register your delete function, using the function objectstore::set_transient_delete_function(). For more information, see objectstore::set_transient_delete_function() in the C++ API Reference.
Examples
The following example releases the storage referenced by the pointer obj:
delete obj;You also can use ::operator delete() to release storage allocated for arrays by including brackets, as provided by the standard C++ syntax. The following example releases storage allocated for an array of objects:
delete [ ] some_objs;
Using Typespecs
When you use ::operator new() to allocate persistent storage, one of the arguments you must supply is a pointer to an os_typespec object; for information about the syntax, see ::operator new(). This argument, called a typespec, identifies the type and layout of the allocation.
There are three ways to get a typespec:
Typespecs for Fundamental Types
ObjectStore provides special functions for retrieving typespecs for the fundamental types that are built into the C++ language, such as char and int. These functions are static members of the class os_typespec. The first time such a function is called by a process, it allocates the typespec and returns a pointer to it. Subsequent calls to the function in the same process do not result in further allocation; instead, a pointer to the same os_typespec object is returned.
Example
The following example illustrates how to use os_typespec::get_char() to pass a typespec to ::operator new() when allocating persistent storage for a character array. Following is the class declaration:
class Part {
public:
char *descrip;
int id;
Part(char* s, int n);
};
Following is the definition of the constructor that allocates the array:
Part::Part(char* s, int n)
{
int len = strlen(s)+1;
descrip = new(os_cluster::with(this),
os_typespec::get_char(), len) char[len];
strcpy(descrip, s);
id = n;
}
The first argument to new() clusters the allocation for the string with the allocation for the Part object (see Clustering). The second argument supplies the typespec by calling os_typespec::get_char(). The third argument supplies the number of elements in the array.
Typespecs for Classes
If you can make changes to source code and add a member to a class, the simplest way to retrieve a typespec for that class is to add the following member function to the class declaration:
static os_typespec *get_os_typespec();You can use any appropriate access specifier (public, private, or protected) for your application. The ObjectStore schema generator (ossg) implements this function by supplying a body automatically. At run time, you can call your_class::get_os_typespec() for a pointer to a typespec for your_class. The return value is static and reusable; your application incurs no expense by calling it and does not have to delete the return value.
For reference information about get_os_typespec(), see os_typed_pointer_void in the C++ API Reference.
Following is the class definition of Part, now modified from the version listed in Typespecs for Fundamental Types, to use get_os_typespec():
class Part {
public:
char *descrip;
int id;
static os_typespec *get_os_typespec();
Part(char* s, int n);
};
The following statement uses Part::get_os_typespec() to supply a typespec to ::operator new():
Part *a_part = new(db, Part::get_os_typespec()) Part("wrench", 111);
The first argument, db, is the where argument that specifies the database where a_part will be stored. The second argument is the typespec, supplied by the call to Part::get_os_typespec(). For other examples of Part::get_os_typespec(), see the programs listed in Step 2. Associating the Root with an Entry-Point Object and Step 3. Retrieving the Entry-Point Object.
If it is not possible to change the source code by adding get_os_typespec() as a member function of a class, you must construct an os_typespec object, as described in the next section.
Constructing a Typespec
The simplest way to get a typespec is to call one of the member functions of the class os_typespec or to add get_os_typespec() as a member function of the class for which you need the typespec. Both approaches are described in Typespecs for Fundamental Types and Typespecs for Classes, respectively.
os_typespec::os_typespec(char* "type_name");type_name is the name (in quotation marks, no embedded spaces) of a class or fundamental type (such as int and char). See the description of os_typed_pointer_void in the C++ API Reference for detailed information.
The following statement constructs a typespec:
os_typespec *part_ts = new os_typespec("Part");
After you have created an os_typespec for a particular type (the type emp_typespec in the example above), you can use it repeatedly to create all of the program's new instances of that type.
// legal but inefficient
for (int i=1; i < 100000; i++) {
os_typespec part_ts("Part");
new(db, &part_ts) Part (i);
}
Following is the preferable way to code this loop:
os_typespec part_ts ("Part");
for (int i=1; i<100000; i++)
new(db, &part_ts) Part (i);
template <class T> class PT {
. . .
void foo(os_database *db) {
// allocate PT<T> persistently
new(db, "PT<???>") PT<T>();
// allocate type T persistently
new(db, "???") T();
}
};
This method does not work, however, because there is no way when you are coding to know the information to fill in for ???. T cannot be used because the C++ template facility does not instantiate it properly. It is inside a string. The solution is to declare a get_os_typespec() member function for the parameterized class and for each class that will serve as parameter.
class PT_parm {
public:
static os_typespec *get_os_typespec();
};
template <class T> class PT {
static os_typespec *get_os_typespec();
void foo(os_database *db) {
// allocate new PT<T> persistently
new(db, PT<T>::get_os_typespec()) PT<T>();
// allocate type T persistently
// (assuming it is a suitable class)
new(db, T::get_os_typespec()) T();
}
};
Consider, for example, the following class declaration:
class A_class {
// some members
...
char* s;
};
If an A_class object has been allocated persistent storage and you intend to allocate some persistent storage for s, clustering would require allocating s as close as possible to the allocation for the A_class object. You achieve optimal clustering in ObjectStore applications by allocating related objects in the same cluster. (As explained in Persistent Storage: Organizational Overview, a cluster is the basic physical unit of allocation in an ObjectStore application.) ObjectStore has overloaded ::operator new() to give you control over the placement of objects in persistent storage. The two that are relevant to clustering are
void* ::operator new ( os_cluster::of(obj), typespec ) void* ::operator new ( os_cluster::with(obj), typespec )obj is a pointer to a previously allocated object that you want clustered with the new allocation. Both methods are static, so you do not need an instance of os_cluster to call them.
The difference between them is that the os_cluster::of() overloading of new allocates storage in the same cluster as obj, whereas the os_cluster::with() overloading allocates as close as possible to obj. os_cluster::with() returns an automatically allocated os_cluster_with object. As explained in os_cluster::with() and os_cluster_with in the C++ API Reference, this object contains a pointer to the cluster in which obj is stored. The effect of the with() overloading, however, is to allocate as close as possible to obj.
With either overloading, if the cluster is full (the maximum size of a cluster is 2 GB) or contains a huge allocation (greater than 64 KB), the allocation fails and ObjectStore signals err_cluster_full. If the allocation is huge, ObjectStore creates a huge cluster in the same segment as the specified cluster of obj and allocates from that cluster.
Note
The os_cluster::of() method used to overload ::operator new() returns a pointer to a transient os_cluster object that should be deleted when no longer needed. For information about managing the lifetimes of this and other transient objects, see Managing Lifetimes of ObjectStore Objects. For information about os_cluster::of(), see the C++ API Reference.
Pointers and Persistent Memory
Using pointers in an ObjectStore application is in many ways no different from using them in a program that works only with transient memory. When you use ::operator new() to allocate persistent storage, you can use the pointer it returns in much the same way that you use pointers to transient memory. However, there are some precautions and restrictions regarding the use of persistent pointers. The following sections explain under what circumstances you can and cannot use persistent pointers, and how to control ObjectStore's checking for illegal pointers. Persistent Pointers in Non-ObjectStore Processes
It is illegal to pass a pointer to persistent memory to a non-ObjectStore process.
For reference information about os_with_mapped, see os_with_mapped in the C++ API Reference.
{
os_with_mapped mybuf(persistent_buffer, buffer_size, 1);
read(fd, persistent_buffer, buffer_size);
}
The constructor for os_with_mapped ensures that the necessary pages are available and mapped with appropriate access rights, and that those pages are wired into the client cache until the destructor is run. The constructor signals an exception if you run out of room in the cache, if you attempt to wire a page more than 250 times, or if obtaining a page fails because of deadlock. The destructor allows the pages to be moved out of the cache.
Object Design recommends using os_with_mapped whenever you pass a persistent pointer to system calls or library functions that might call system calls or handle memory access violations. There is little overhead to os_with_mapped, so calling it frequently does not degrade performance.
Mapping large ranges, having many simultaneous mappings, or keeping mappings in effect for a long time is not recommended, because this can interfere with cache replacement and lead to a cache-is-full exception.
Cross-Transaction Pointers
Two types of pointers are normally valid only while in a transaction:
For information about retaining transient-to-persistent pointers across transaction boundaries, see the following sections in Chapter 1 of the Advanced C++ API User Guide:
static void set_null_illegal_pointers(os_boolean);Setting the argument to 0 (false) specifies the default behavior - signal an exception. The change in behavior is in effect only for the current session.
To retrieve the current behavior, call the following method:
static os_boolean get_null_illegal_pointers();Both set_null_illegal_pointers() and get_null_illegal_pointers() are defined by four classes, as described in the C++ API Reference:
Correcting illegal pointers
When you are debugging a database, an illegal pointer might already have been written to the database that you want to correct. To prevent ObjectStore from signaling an exception when it detects an illegal pointer while relocating pages from the database to the client cache, call the following method:
static void objectstore::set_always_null_illegal_pointers(
os_boolean);
Calling this method allows you to correct an illegal pointer. Note that this method is defined by the objectstore class only. For more information, see objectstore::set_always_null_illegal_pointers() in the C++ API Reference.
Other illegal pointers
Other illegal pointers that can corrupt a database include
| Class | Level of Granularity |
|---|---|
| objectstore | All databases in the current session |
| os_database | A specific database |
| os_segment | A specific segment in a database |
| os_cluster | A specific cluster in a segment |
For detailed information about these methods and the settings they retrieve or specify, see Chapter 2, Class Library, of the C++ API Reference. For information about using get_null_illegal_pointers() and set_null_illegal_pointers(), see Illegal Pointer Checking.
The following illustrates how these methods work when specified at different levels:
pseg->set_fetch_policy(os_fetch_cluster);If the fetch policy for the database containing this segment was previously set to os_fetch_segment, the call would override that setting only for the segment referenced by pseg. If you were subsequently to call the os_database::set_fetch_policy() method, the new policy would apply to all segments in the database, including the one referenced by pseg.
Note that calling the objectstore::set_fetch_policy() method overrides the fetch policy currently in effect for all databases in the current session, including all segments and clusters in those databases.
For information about fetch policy, see Setting Data Fetch Policies in Chapter 1 of the Advanced C++ API User Guide.
Grouping Clusters in Segments
The segment is the unit of persistent storage represented by the class os_segment. (To see how the segment fits in the hierarchical organization of persistent storage, refer to Persistent Storage: Organizational Overview.) If the cluster is the level for grouping objects for optimum performance (as described in Clustering), the segment is the level for grouping clusters that are logically related. That is, segments allow you to group clusters (and the objects they contain) according to certain shared characteristics.
Segment-level access
For example, you might want to restrict access to one group of related objects in a database and not to others. Preparatory to setting any permissions, you would create a "restricted" segment, allocate the objects from this segment, then use the os_segment_access class to establish appropriate access permissions. For information about using the Segment-Level Access API, see Controlling Segment-Level Access.
Segment reference policies
You can also specify segment reference policies at the segment level. Segment reference policies allow highly efficient segment-level data reorganizations (for example, garbage collection and compaction) that do not affect pointers to the data from other segments. For more information, see Setting Segment Reference Policies in Chapter 1 of Advanced C++ API User Guide.
Using Segment Comments
Comments are strings that you write to or read from a segment. They are persistently stored with the segment, and provide a useful and humanly readable way to identify its contents. When you use the ossize utility to display information about a database, its output includes any segment comments.
char* os_segment::set_comment() const; void os_segment::get_comment(const char* comment );comment is a null-terminated string of indefinite length.
Comments are applicable only at the segment level. For more information, see the following in the C++ API Reference:
For more information about the ossize utility, see ossize: Displaying Database Size in Managing ObjectStore.
Managing Lifetimes of ObjectStore Objects
Certain methods in the ObjectStore API return pointers to transient objects (ObjectStore objects), including objects of the following classes:
Some ObjectStore objects are easily managed because they are seldom needed except for a well-defined purpose. For example, an os_server object is usually created for a single purpose and can be deleted after the purpose has been achieved.
Other ObjectStore objects are more frequently used but have a clearly marked period of usefulness. For example, an os_transaction object is generally used for managing a dynamic transaction. After the transaction has committed, the object can be deleted.
For objects of the classes os_cluster, os_database, os_database_root, and os_segment, it is less clear when to delete them. Because of the special relationship between these objects and the persistent entities they represent, an application is likely to use them frequently and repeatedly, and in different parts of the application. One part of an application can pass an ObjectStore pointer to another part and not know when it can safely delete the object.
The following sections provide information about managing the lifetimes of ObjectStore objects of the classes os_cluster, os_database, os_database_root, and os_segment:
To get the benefit of the use count for help in managing ObjectStore objects, the application must use the Retain-Release API, as described in the next section.
In the following example, db points to a database for which the code retrieves a pointer to an os_segment object that represents the default segment:
os_segment* pseg = db->get_default_segment(); . . . // operations on the segment referenced by pseg . . . pseg->release_pointer();If pseg is the only outstanding pointer returned against the default segment, the call to release_pointer() deletes the object; otherwise, it decrements the count of pointers to this object that the application is currently using.
Calling retain_pointer() is also useful in multithreaded applications when one thread (Thread1) passes an ObjectStore pointer to another (Thread2). Thread2 would call retain_pointer() so that Thread1 could release the pointer without the risk of releasing Thread2's copy. Thread2 would call release_pointer() when it finishes with the copy.
When you use delete on an ObjectStore object, it deallocates the object without regard to its use count. It is, therefore, essential that before deleting an object, your application know that the object is no longer used anywhere else in the application. After the object is deleted, any pointers to the object are invalid.
Releasing Pointers with os_release_ . . . _pointer Classes
ObjectStore provides the classes listed in the following table for releasing pointers to the indicated ObjectStore objects:
| Class | Pointer type |
|---|---|
| os_release_cluster_pointer | os_cluster |
| os_release_database_pointer | os_database |
| os_release_root_pointer | os_database_root |
| os_release_segment_pointer | os_segment |
Each class enables you to release a pointer when a variable of the class goes out of scope. After obtaining an ObjectStore pointer, you declare an automatic variable of the appropriate class and pass the pointer to the variable's constructor. When the variable goes out of scope, its destructor releases the pointer.
Using a os_release_. . . _pointer class to release a pointer respects an application's modularity. It allows one part of an application to obtain and release a pointer without having to know if the object it points to is still in use elsewhere in the application.
{
os_cluster* pc = seg->get_default_cluster();
os_release_cluster_pointer auto_var(pc);
. . .
// operations on the cluster
. . .
} // auto_var's destructor releases pc
For more information about get_default_cluster(), see os_segment::get_default_cluster() in the C++ API Reference.
Methods Returning Pointers to ObjectStore Objects
The following methods return pointers to ObjectStore objects and are likely to be called frequently and repeatedly:
for ( . . . ) {
new_obj = new(os_cluster::of(old_obj), obj_type) . . . ;
. . .
}
The trouble with this loop is that, at each iteration, the call to os_cluster::of() returns a pointer that the application discards, possibly creating a memory leak. How serious this problem is depends on several factors. If old_obj points to an object that is in the same cluster at each iteration, the resulting leak is minor. However, if old_obj points to an object in a different cluster at each iteration, many different os_cluster objects are returned and the leak could be severe, especially if the number of clusters is large. There are several ways to handle this problem. One is to assign the return value from os_cluster::of() explicitly, then call os_cluster::release_pointer(), as follows:
for ( . . . ) {
os_cluster* diff_cluster = os_cluster::of(old_obj);
new_obj = new(diff_cluster, obj_type) . . . ;
. . .
diff_cluster->release_pointer(); // release the os_cluster pointer
}
Another approach is to leave the allocation loop as originally coded and arrange to delete the os_cluster objects in some other part of your program. This approach only works if your program can determine which os_clusters were returned and when it is safe to delete them.A third approach is to avoid the creation of os_cluster objects by using the os_cluster::with() overloading of ::operator new(). os_cluster_with() returns an automatic variable that is destroyed whenever it goes out of scope.
Following is the same example, coded to use os_cluster::with():
for ( . . . ) {
new_obj = new(os_cluster::with(old_obj), obj_type) . . . ;
. . .
}
For more information about using os_cluster::with(), see Clustering.
get_all...() methods
Other methods that risk memory leaks include the get_all. . .() methods defined by os_database and os_segment, for example, os_segment::get_all_clusters(). These methods set up a table of pointers, each of which points to an ObjectStore object that, if not deleted, can stay allocated for the life of the application. For more information, see the following in C++ API Reference:
Updated: 03/09/99 13:54:18