- •Contents
- •Notices
- •Trademarks
- •Preface
- •The team who wrote this book
- •Now you can become a published author, too!
- •Comments welcome
- •Stay connected to IBM Redbooks
- •Chapter 1. Introduction
- •1.1 The opportunity of the in-memory database
- •1.1.1 Disk databases cannot expand to memory
- •1.1.2 IBM solidDB IMDB is memory-friendly
- •1.1.3 Misconceptions
- •1.1.4 Throughput and response times
- •1.2 Database caching with in-memory databases
- •1.2.1 Databases are growing
- •1.2.2 Database caching off-loads the enterprise server
- •1.2.3 IBM solidDB Universal Cache
- •1.3 Applications, competition, and the marketplace
- •Chapter 2. IBM solidDB details
- •2.1 Introduction
- •2.2 Server architecture
- •2.2.1 Database access methods and network drivers
- •2.2.2 Server components
- •2.3 Data storage in solidDB
- •2.3.1 Main-memory engine
- •2.4 Table types
- •2.4.1 In-memory versus disk-based tables
- •2.4.2 Persistent versus non-persistent tables
- •2.4.3 Choosing between different table types
- •2.5 Transactionality
- •2.5.1 Concurrency control and locking
- •2.5.2 Isolation levels
- •2.5.3 Durability levels
- •2.6 solidDB SQL extensions
- •2.6.1 solidDB SQL standard compliance
- •2.6.2 Stored procedures
- •2.6.3 Triggers
- •2.6.4 Sequences
- •2.6.5 Events
- •2.6.6 Replication
- •2.7 Database administration
- •2.7.1 Configuration settings
- •2.7.2 ADMIN COMMAND
- •2.7.3 Data management tools
- •2.7.4 Database object hierarchy
- •Chapter 3. IBM solidDB Universal Cache details
- •3.1 Architecture
- •3.1.1 Architecture and key components
- •3.1.2 Principles of operation
- •3.2 Deployment models
- •3.3 Configuration alternatives
- •3.3.1 Typical configuration
- •3.3.2 Multiple cache nodes
- •3.3.3 SMA for collocation of data
- •3.3.4 solidDB HSB servers for high availability
- •3.4 Key aspects of cache setup
- •3.4.1 Deciding on the replication model
- •3.4.2 Defining what to replicate
- •3.4.3 Starting replication
- •3.5 Additional functionality for cache operations
- •3.5.1 SQL pass-through
- •3.5.2 Aging
- •3.5.3 Improving performance with parallelism
- •3.6 Increasing scale of applications
- •3.6.1 Scaling strategies
- •3.6.2 Examples of cache database applications
- •3.7 Enterprise infrastructure effects of the solidDB Universal Cache
- •3.7.1 Network latency and traffic
- •3.7.3 Database operation execution
- •Chapter 4. Deploying solidDB and Universal Cache
- •4.1 Change and consideration
- •4.2 How to develop applications that use solidDB
- •4.2.1 Application program structure
- •4.2.2 ODBC
- •4.2.3 JDBC
- •4.2.4 Stored procedures
- •4.2.5 Special considerations
- •4.3 New application development on solidDB UC
- •4.3.1 Awareness of separate database connections
- •4.3.2 Combining data from separate databases in a transaction
- •4.3.3 Combining data from different databases in a query
- •4.3.4 Transactionality with Universal Cache
- •4.3.5 Stored procedures in Universal Cache architectures
- •4.4 Integrate an existing application to work with solidDB UC
- •4.4.1 Programming interfaces used by the application
- •4.4.2 Handling two database connections instead of one
- •4.5 Data model design
- •4.5.1 Data model design principles
- •4.5.2 Running in-memory and disk-based tables inside solidDB
- •4.5.3 Data model design for solidDB UC configurations
- •4.6 Data migration
- •4.7 Administration
- •4.7.1 Regular administration operations
- •4.7.2 Information to collect
- •4.7.3 Procedures to plan in advance
- •4.7.4 Automation of administration by scripts
- •Chapter 5. IBM solidDB high availability
- •5.1 High availability (HA) in databases
- •5.2 IBM solidDB HotStandby
- •5.2.1 Architecture
- •5.2.2 State behavior of solidDB HSB
- •5.2.3 solidDB HSB replication and transaction logging
- •5.2.4 Uninterruptable system maintenance and rolling upgrades
- •5.3 HA management in solidDB HSB
- •5.3.1 HA control with a third-party HA framework
- •5.3.2 HA control with the watchdog sample
- •5.3.3 Using solidDB HA Controller (HAC)
- •5.3.4 Preventing Dual Primaries and Split-Brain scenarios
- •5.4 Use of solidDB HSB in applications
- •5.4.1 Location of applications in the system
- •5.4.2 Failover transparency
- •5.4.3 Load balancing
- •5.4.4 Linked applications versus client/server applications
- •5.5 Usage guidelines, use cases
- •5.5.1 Performance considerations
- •5.5.2 Behavior of reads and writes in a HA setup
- •5.5.3 Using asynchronous configurations with HA
- •5.5.4 Using default solidDB HA setup
- •5.5.5 The solidDB HA setup for best data safeness
- •5.5.6 Failover time considerations
- •5.5.7 Recovery time considerations
- •5.5.8 Example situation
- •5.5.9 Application failover
- •5.6 HA in Universal Cache
- •5.6.1 Universal Cache HA architecture
- •5.6.2 UC failure types and remedies
- •6.1 Performance
- •6.1.1 Tools available in the solidDB server
- •6.1.2 Tools available in InfoSphere CDC
- •6.1.3 Performance troubleshooting from the application perspective
- •6.2 Troubleshooting
- •Chapter 7. Putting solidDB and the Universal Cache to good use
- •7.1 solidDB and Universal Cache sweet spots
- •7.1.1 Workload characteristics
- •7.1.2 System topology characteristics
- •7.1.3 Sweet spot summary
- •7.2 Return on investment (ROI) considerations
- •7.2.1 solidDB Universal Cache stimulates business growth
- •7.2.2 solidDB server reduces cost of ownership
- •7.2.3 solidDB Universal Cache helps leverage enterprise DBMS
- •7.2.4 solidDB Universal Cache complements DB2 Connect
- •7.3 Application classes
- •7.3.1 WebSphere Application Server
- •7.3.2 WebLogic Application Server
- •7.3.3 JBoss Application Server
- •7.3.4 Hibernate
- •7.3.5 WebSphere Message Broker
- •7.4 Examining specific industries
- •7.4.1 Telecom (TATP)
- •7.4.2 Financial services
- •7.4.3 Banking Payments Framework
- •7.4.4 Securities Exchange Reference Architecture (SXRA)
- •7.4.5 Retail
- •7.4.6 Online travel industry
- •7.4.7 Media
- •Chapter 8. Conclusion
- •8.1 Where are you putting your data
- •8.2 Considerations
- •Glossary
- •Abbreviations and acronyms
- •Index
2.5 Transactionality
IBM solidDB guarantees reliable transactional processing by implementing a database server that satisfies all ACID (atomicity, consistency, isolation, durability) requirements.
Atomicity requires that database modifications must follow an “all or nothing” rule. Each transaction is said to be atomic. If one part of the transaction fails, the entire transaction fails and the database state is left unchanged.
Consistency ensures that any transaction that the database performs can take it from one consistent state to another.
Isolation refers to the requirement that other operations cannot access data that has been modified during a transaction that has not yet completed. The question of isolation occurs in case of concurrent transactions (multiple transactions occurring at the same time).
Durability is the ability of the DBMS to recover the committed transaction updates against any kind of system failure (hardware or software). Durability is the DBMS guarantee that after the user has been notified of a transaction's success, the transaction will not be lost.
2.5.1Concurrency control and locking
The purpose of concurrency control is to prevent two users (or two connections by the same user) from trying to update the same data at the same time. Concurrency control can also prevent one user from seeing uncommitted (dirty) data while another user is in the process of updating it.
More generally, concurrency control is used to preserve the overall correctness
of concurrent transaction executions. The ultimate form of that correctness is called serializability. A serializable execution of concurrent transactions
produces a result that is identical to a case when all these transaction would be executed serially: one after another. Preserving generalized serializability for all
possible cases is resource-consuming. Therefore, the actual correctness can be set with a parameter called isolation level that can be adjusted as needed, even
dynamically.
Chapter 2. IBM solidDB details 27
IBM solidDB offers two concurrency control mechanisms, pessimistic concurrency control and optimistic concurrency control:
Pessimistic concurrency control mechanism is based on locking. A lock is a mechanism for limiting other users' access to a piece of data. When one user has a lock on a record, the lock prevents other users from changing (and in some cases reading) that record.
Optimistic concurrency control mechanism does not place locks but prevents the overwriting of data by using timestamps.
D-tables are by default optimistic; M-tables are always pessimistic. In D-tables, you can override optimistic concurrency and specify pessimistic locking instead. You can do this at the level of individual tables. One table might follow the rules of optimistic concurrency while another table follows the rules of pessimistic locking. Both tables can be used within the same transaction and even the same statement; solidDB handles this internally.
Pessimistic concurrency control
Pessimistic concurrency control (or pessimistic locking) is called pessimistic because the system assumes the worst; it assumes that two or more users will want to update the same record at the same time, and then prevents that possibility by locking the record, no matter how unlikely conflicts actually are.
The locks are placed as soon as any piece of the row is accessed, making it impossible for two or more users to update the row at the same time. Depending on the lock mode (shared, exclusive, or update), other users might be able to read the data although a lock has been placed.
Optimistic concurrency control
Optimistic concurrency control assumes that although conflicts are possible, they will be rare. Instead of locking every record every time that it is used, the system merely looks for indications that two users actually did try to update the same record at the same time. If that evidence is found, then one user’s updates are
discarded and the user is informed. The step of checking whether a transaction can commit is called transaction validation. Typically, the validation is performed
at the commit time but solidDB uses, by default, a modified method called early validation. With early validation, the data being read (the read-set) and written
(write-set) are checked against other transactions at each database operation, without waiting for commit. If the data in the read-set and write-set has changed since the beginning of the transaction, the transaction is considered to be violating the data consistency and is aborted. The details of checking rules for read-sets and write-sets depend on the isolation level that will be discussed shortly.
Optimistic concurrency control is available for disk-based tables (D-tables) only.
28 IBM solidDB: Delivering Data with Extreme Speed
Locking and performance
Optimistic concurrency allows fast performance and high concurrency (access by multiple users), at the cost of occasionally refusing to write data that was initially accepted but was found at the last second to conflict with another user's changes.
Pessimistic locking requires overhead for every operation, whether or not two or more users are actually trying to access the same record. The overhead is small but adds up because every row that is updated requires a lock. Furthermore, every time that a user tries to access a row, the system must also check whether the requested row or rows are already locked by another user or connection.
When using M-tables, best performance in reference to locking is achieved with temporary tables; because concurrent users cannot access data in temporary tables, temporary tables do not use concurrency control.
When using D-tables, optimistic concurrency control provides the best performance, if the possibility for conflicts is low. For example, there are many records but relatively few users, or few updates and mostly read-type operations. However, in workloads that expose more updates, like in typical online transaction processing (OLTP) applications, pessimistic locking is actually more beneficial.
2.5.2 Isolation levels
solidDB supports the isolation levels defined in the SQL-92 standard, except for the READ UNCOMMITTED level2. The isolation level can be set per session or per statement.
The three supported isolation levels are explained in the following sections.
READ COMMITTED
This isolation level allows a transaction to read only committed data. However, the view of the database may change in the middle of a transaction when other transactions commit their changes.
In solidDB HotStandby configurations, the isolation level of the Secondary server is always READ COMMITTED.
2Uncommitted, or dirty, reads violate transactional paradigm as modified data becomes visible before the transaction performing the update completes. Hence the READ UNCOMMITTED isolation level is rarely, if ever, used in production database systems.
Chapter 2. IBM solidDB details 29
REPEATABLE READ
This isolation level allows a transaction to read only committed data and
guarantees that read data will not change until the transaction terminates. With optimistic D-tables, solidDB, in fact, maintains an isolation model called snapshot isolation. It ensures that the transaction sees a consistent view of the database.
When using optimistic concurrency control, conflicts between transactions are detected by using only the transaction write-set validation. For example, if a transaction involves one read and one update, solidDB validates that no one has updated the same row in between the read operation and the update operation. In this way, lost updates are detected, but the read is not validated. With transaction write-set validation only, transactions are not serializable in many cases. Additionally, phantoms may occur. Phantoms are table rows that appear (are seen) in the course of the transaction although were not seen in the beginning. Such rows may result from insert and update activities by other concurrent transactions.
In M-tables, the repeatable read level is implemented in a more traditional level—by way of locks. Shared locks are kept on all the data items read until the transaction commit. That preserves other transactions from changing the read-set. The correctness of the write-set is assured with exclusive locks on all the items written. Because both read-set a d write-set are protected, transactions running on M-tables, in the repeatable read mode, are serializable with the exceptions of phantoms.
SERIALIZABLE
This isolation level allows a transaction to read only committed data with a consistent view of the database. Additionally, no other transaction may change the values read by the transaction before it is committed because, otherwise, the execution of transactions cannot be serialized in the general case.
With D-tables, solidDB can provide serializable transactions by detecting all conflicts between transactions. It does this by using both write-set and read-set validations. Because no locks are used, all concurrency control anomalies are avoided, including the phantoms.
The SERIALIZABLE isolation level is not supported with M-tables.
2.5.3 Durability levels
The durability level controls how solidDB handles transaction logging. The solidDB server supports three durability levels: strict, relaxed, and adaptive. Relaxed durability yields best performance; strict durability minimizes loss of transactions. The adaptive durability level is available only in HotStandby configurations.
30 IBM solidDB: Delivering Data with Extreme Speed
