Distributed Caching/Database Offload with Terracotta DSO

You've probably heard about how Terracotta DSO (Terracotta's flagship general purpose JVM-level clustering technology) can help you offload your database. You have a variety of choices depending on your implementation and your goals - This article tries to bring them all together in 1 place so you could more easily determine what Terracotta offering better suits your needs.

See the attached diagram for a decision matrix of sorts. The Green Lozenges lead you to products on terracotta.org that might best fit your stack.

In a nutshell:

• At a high-level, the first step is to acknowlede the problem with the implementation.
• This problem will typically manifest itself as UNACCEPTABLE LATENCY associated with certain application transactions OR as DATABASE RESOURCES Being PEGGED (e.g. CPU running uncomfortably high, for the DBA/SysAdmin's comfort).
• Now, it follows that if the data the App is mutating is "close" to the application then that is much better from a latency perspective, than if the data has to be fetched off the network from your SGA/Disk that comprises your Oracle Database Instance. This data may rightfully live on the database but it will still greatly improve latency if all operations were done in memory against pre-fabricated Domain Model Objects. Enter CACHING. Now your application presumably runs on a cluster of JVMs - so just caching locally on a given JVM, would render the caches inconsistent amongst the JVMs, if these caches and or the Database are also receiving updates to the underlying data. Enter DISTRIBUTED CACHING - where all Read and Write access now occurs in Memory across all the cluster JVMs (and is kept in synch with the database - by "writing-through" or "writing-behind"). i.e. Decide if you just need Caching (e.g. read only data set), or you need Distributed Caching ( a data-set that is being modified by the application as well. A Distributed Cache for a read-only data-set only helps in reducing query load to the database by 1/N where N is the number of client-JVMs in the cluster).
• Once you've established that Distributed Caching is the solution - decide on whether changes need to be persisted to the Database, and if so, need to be written synchronously (write through) or asynchronously (write behind).
  
• Decide on whether the application can tolerate any incorrectness amongst threads on different JVMs reading/writing to the cache. Terracotta is really the only solution, in my opinion, if the application cannot tolerate ANY incoherent concurrent access, whilst still delivering scale/performance.
  

In summary:
A> There are several technologies that enable "DISTRIBUTED CACHES". So in that usage - yes, Terracotta is one of many such "distributed cache" products in the market - but Terracotta's Networked Attached Memory innovation and implementation offer some unique and powerful advantages:

1. Preservation of your Java Programming Model - you write natural Java - no new API to program to. You could thus cluster your own custom caches be they implemented as Hashmaps, Hastables, ConcurrentHashMaps, LinkedHashMaps etc.
   
2. Correctness guarantees (as defined by the Java Memory Model) of access across the cluster of participating JVMs.
   
3. Ready Made clustering solutions (Terracotta Integration Modules - TIMs) for popular OSS caches (e.g. EHcache etc.).
4. Ready Made support for ORMs (Terracotta Integration Modules - TIMs): Hibernate (Detached-Mode, 2nd Level Cache); Ibatis etc.
5. "Intelligent replication" to minimize Network Chatter and Network Replication payloads (byte code injection gurantees only mutated Object Fields are shipped to at least 1 other element in the network - the Terracotta Server (and more than 1 if other elements in the cluster reference that field)).
6. Abilitity for the cache to exceed the size of HEAP on a given application JVM without the need for partitioning (assuming somewhat Gaussian access patterns across key-space).
7. In depth Monitoring and Visibility via the Developer Console and the Terracotta Operations Center.
8. L1 (component in client JVM), L2 (dedicated Terracotta JVM) design with persistence affords higher Level of HA - e.g. cache preserved even if all JVMs are lost. HA is in the box - in that no Shared Disk be provisioned.
   
9. JMX Events and Hooks to respond programatically to cluster situations such as Client JVM failure or new Client JVM introduction.
   
10. OSS Model of Distribution and many others.....

In addition,
B> Evaluate if certain data sets in the database really needs to be persisted in the database or if the persistence afforded by Terracotta suffices. Typically - data should be in your RDBMS/System of Record

• If it is long-lived and business-critical. See http://tech.puredanger.com/2008/08/01/thinking-about-data-lifetimes/
• If the Data need to be Reported on?
• If the Data need to be queried extensively along many diverse criteria - SQL is much more expressive than OQL or any such option? (e.g. if you have an entity X with 3 attributes and Y with 4 attributes where 1 attribute is common across them then you can query X alone in 3C1 + 3C2 + 3C3 = 7 ways and Y alone in 4C1 + 4C2 + 4C3 + 4C4 = 15 ways and X & Y together in multiple other ways).

In cases other than the ones mentioned above - there is a strong possibility that you could just work off memory, assuming some technology like Terracotta gave you HA for that chunk of memory. Advantages apart from DB Offload is the simpler code base - since one simply works off the Java Domain Model, where Terracotta delivers HA and correctness of access for those data structures, without the need to involve a Database and the marshalling/unmarshalling code to overcome the impedance mismatch between the object world and the relational world. This has been proven out on several occasions now - you can get more at http://www.terracotta.org/web/display/orgsite/Common+Use+Patterns

So hopefully after reading this, it is easier for you to determine which Terracotta offering best solves your data-base offload problem....

Tuning, Testing, Deploying Terracotta Implementations - how does my application time-to-market change?

I work at Terracotta with the Field-Engineering Team. What we have noticed over 100s of implementations, is that a lot of technology-adaptors thus far, are pretty adept at getting to Dev Completion with Terracotta-DSO on their own. This coupled with the Terracotta Integration Module strategy (see http://www.terracotta.org/confluence/display/integrations/Home ) makes me believe that Terracotta’s promise of maintaining natural Java Semantics and hence reducing the level of effort needed by the Developer to cluster his/her app is, by and large, very real.

Another observation though is that, self-adopters while successful at getting to Integration-Complete phases of their projects, have needed, relatively speaking, more hand-holding with regards to tuning, testing and deploying their applications into production. Reasons are:

• For the most part, programming languages feature no specification of behavioral characteristics outside of the contractual interface specification of inputs and resultant outputs. As an example, let's take the notion of sizing collections. We all understand that one should pre-size collections, so as to avoid expensive rehashing at run-time. At a default of 16 and load facotr of .75, as soon as the size of the Hashtable gets to 12, we are already paying the cost of a rehash (to roughly 32).But very few developers actually size their collections due to whatever reason (ignorance, negligence, or genuine inability to predict the collection-size). It becomes a later mop-up tuning task, as against being accounted for during implementation....Now a Hashtable/Hashmap with some self-correcting size related heuristics or some eventing on rehash (so one can react to it programmatically) would be useful - but in the absence of one, there is no substitute to competence when implementing.
• The above argument is to highlight that even with technologies (such as Terracotta) that modify byte-code to give you desirable characteristics of add-a-node-predictive-scale and High Availability, there is really no substitute for thought and discipline when engineering, rigor during testing and scientific methodologies when stress-testing and deploying.
• Of course, Terracotta is a byte-code enhancing technology and is working on providing the end-user more tooling, more visibility and more rules/inference-engines to suggest possible root-causes – and hence ease the process across the entire SDLC. You already get the gist of this approach via the Snapshot Visualization Tool and there is a lot more, coming down the pipe.

So then assuming that you have had to make some delta-changes to your application to get to Dev-Completion (e.g. initialize Terracotta transients , modify the coarse-ness of locks etc.) what is the typical time spent around Tuning, Testing, Deploying ? Perhaps these comparisons between any Java Application and a Terracotta-integrated Java Application can highlight the delta involved - this way you could have a clearer understanding of how time-to-market for your application changes, when you integrate Terracotta from POC-Production.
(Note that java Performance tuning is such a vast topic – and so we aren’t talking about all possible ways to enhance performance/scale of your Java app itself, so no NIO or AWT improvements for example - just identifying what one typically does during Terracotta tuning and how really a fair bit of it is similar to what one would do, without Terracotta in the picture, in the first place.)

Tuning Terracotta: See here for several common practices:

Programmer Consideration		Plain Old Java Application (POJA)	Terracotta integrated Plain Old Java application (TI-POJA)
Data structures	Size	Validate right choices of Data structure based on expected size of the structure.	Similar to a POJA. Additional Considerations: · Whether a collection is “partial” or not (Hashmap, CHM, Hashtable, LinkedHashMap, Arrays are partial – others are not as of 2.6). · Does the object-graph play well with the Terracotta Virtual Memory Manager
	Pattern of Access	If lookup based on key use associative array (e.g. Hashmap) instead of a Vector etc.	Identical to a POJA
	Extent of I/O against the collection and Concurrency of Access	Example: · Hashtable or Hashmap based on correctness needs? · Hashmap or ConcurrentHashMap based on how concurrent the access needs to be?	Same concept how concurrent is the access across the cluster.
	Data-structure implementation	Myriad other considerations based on how the data-structure is implemented and the domain-model under consideration. See here for more discussion	Identical to a POJA
Synchronization (given a requirement of a certain level of correctness)	Contention on monitor that you are synchronizing on	Be careful to lock on the right monitor. May need to stripe Lock.	Identical
	Scope of lock	Coarse grained versus Fine grained (is it method level or can you just keep it local to a specific block of code).The more fine-grained the better.	Identical - just note that in most cases there maybe value to making the lock more coarse-grained to benefit from batching.
	Type of lock	Pessimistic Synchronization or lower level synchronization semantics (volatile or Atomic or ReentrantReadWriteLock etc.)	Volatiles are not supported as of 2.6. In addition one has to look at lock-types e.g. Synchronous-Write, Write, Read, Concurrent, None. You can get more details on distributed locking here .
Memory Management	Garbage Collector: Choice of Collector. Xms, Xmx, Ratio of Eden to Old etc.	CMS or Parallel or both. Appropriate sizing of Heap, Young, Old etc.	Identical. See here In addition one needs to consider DGC (Distributed Garbage Collection) on the Terracotta Server.
	Soft References	Soft Reference Policy.	Tune Virtual Memory Manager on L1, L2.
	Virtual Memory Usage.	Avoid thrash with virtual memory monitor via vmstat	Identical – some thrash is allowable as long as within Latency SLA– between L1<->L2<->L2Disk. Tune faulting, fault-flush, committing-Terracotta transactions
Traffic Patterns and Distribution. Locality of Reference		Based on data size - Partition Data across JVMs or not.Locality of reference is a good thing.	Identical (one can over come data-size limitation assuming trade-off on latency SLA is possible). Delta: In several use-cases (but not all) Terracotta VMM can mean that you do not need to partition (hence simpler app). In some cases, given the data-volume and performance SLA, there may be no way to achieve Locality of Reference without partitioning. See here.
Other Considerations			Instrumentation Scope. See here

Testing and Deployment: See here for several thoughts around how testing a distributed application is different from a non-distributed application.

Consideration		Any Plain Old Java Application (POJA)	Any Terracotta integrated Java application (TI-POJA)
Testing	Functional	Code-Coverage is important to determine thoroughness of testing.	Identical - to ensure that there are no possible Terracotta runtime exceptions (UnlockedSharedException and NonPortableExceptions) lurking around.
	Scale/Performance Testing	Need Test Script that mirrors production load. Production Monitoring should also be on. Need quantitative measures (average, median, max) of latencies for a basket of transactions and overall TPS measure.	Testing a distributed app is a different ball-game. Ideally a framework to spin-up/tear-down JVMs is useful. You will t to consider throughput in the presence of various failover scenarios (e.g. when a client-JVM fails, when the Primary Terracotta-server fails etc.), state of distributed components (e.g. a client JVM is at 100% cpu utilization, network switch is in the middle of a re-boot sequence etc.), manifestation of the object graph across client JVMs , Terracotta Server JVMs and disk (given that Terracotta has a Virtual Memory Feature which allows you to exceed physical limits of RAM on your client JVM) etc. You probably also want to run your monitoring scripts at the same time. See here
	Availability	Need to identify Single Points of Failures in the application deployment	Terracotta will provide you an implementation with no single points of failure. Some effort needed towards tuning parameters that determine failover times /behaviors of various components based on the type-of failure. See here. The Network Active/Passive Link also talks about all the availability tests you could execute to reconfirm no SpoF in you specific environment.
Deployment
	Sizing of hosts on which JVM resides	Based on capacity planning exercises	Identical. Additionally need to size Terracotta servers and choose storage strategy for cluster state. You may need to determine how many client JVMs as well to support Application-TPS and based on latency SLAs (i.e. if faulting is not OK)
	Monitoring	CPU, Memory, Disk, Network, GC, TPS of each client-JVM etc.	Identical. In addition probably 10-20 more Terracotta specific parameters to monitor and of course the Terracotta Servers/Disk they write to.
	Run-book	Protocol for Recovery and Troubleshooting in case of situations.	Identical. Given that the architecture will not have single points of failure, the run-book would probably feature more about what to do to restore HA to the environment.

As is evident from the above arguments, there is a fair bit of parallelism between tuning and deploying any Java Application and a Terracotta Integrated App...although distributing your app, necessarily requires a few changes around the entire software-manufacturing process. Hopefully based on the above list and your Dev-exercise, you can predict better the additional time-to-market (I would guesstimate typical ranges average around 3-12 weeks for mid-high complexity apps from start to finish and it will just get better with more TIMs, Visibility and specific Documentation). You can read more about specifics at http://www.terracotta.org/confluence/display/wiki/TechnicalFAQ . We are in the process of cleaning up this documentation with links to supporting product features etc.– so keep an eye out for new content at http://www.terracotta.org