Http Session Externalization onto JBoss Data Grid in OpenShift v3

Vijay Chintalapati bio photo By Vijay Chintalapati Comment

JBoss EAP Http Session Externalization onto JDG in OpenShift v3

Http session replication in a HA (High Availability) EAP profile is relatively well known. A JEE server, such as JBoss EAP, started in a HA profile allows the HTTP session information to be distributed between nodes (EAP servers) of a cluster. Http session replication is very important in webapps particularly when:

  • The webapp starts a session for each user
  • The user then stores 1 or more Java objects within the session (object)
  • The app server nodes of the cluster are behind a Http load-balancer
  • The http sessions are configured to be not sticky at the load-balancer; that is, the next http request after the session is created could go to a different node compared to the node where the session was created (this facilitates the session failover)

How can the use of JDG improve upon the classic setting ?

With the use of a JBoss Data Grid (JDG) cluster, external to the JBoss EAP nodes, the server configuration for HTTP sessions is pointed to the JDG cluster and the HTTP sessions are persisted into a designated cache of the JDG cluster. Using an external JDG cluster as mentioned, the architecture can reap the following benefits:

  • The EAP cluster can be brought down completely (ideally after gradually draining the client connections) without loosing the session information
  • When sessions are dealing with large objects, use of an external JDG cluster will:
    • Keep the individual EAP nodes light and free of heap usage
    • Let the JDG cluster deal with additional storage requirements by scaling horizontally with addition of new JDG server nodes to the existing cluster
  • With both EAP and JDG clusters in Openshift, particularly with Openshift v3.3 scaling of the clusters based on load can be automated

So lets see how we can setup a sample project to demo HTTP session replication with the following products :

  • JBoss EAP 6.4
  • JBoss Data Grid 6.5
  • Openshift 3.2

New Project on OpenShift v3

Log into the OpenShift environment

[root@intrepid origin]# oc login                                      
Authentication required for https://192.168.1.17:8443 (openshift)
Username: admin
Password:
Login successful.

Create a new project

oc new-project http-session-externalization \
   --display-name="HTTP Session Externalization into JDG" \
   --description="Project to demonstrate how to externalize EAP HTTP sessions into a remote JDG cluster"

Create Service Accounts and Assign Correct Permissions

Service accounts are API objects that exist within each project. For processes in containers inside pods to be able to contact the apiserver, they need to be associated with a service account.

The service account eap-service-account should exist for both EAP and JDG and should have a view permission to view the pods.

# Create a service account by the name eap-service-account for EAP 6
oc create serviceaccount eap-service-account -n http-session-externalization

# Assign a view permission to the service account in the current project namespace 
oc policy add-role-to-user view system:serviceaccount:$(oc project -q):eap-service-account -n $(oc project -q)

# Also assing the default service account the view access 
oc policy add-role-to-user view system:serviceaccount:$(oc project -q):default -n $(oc project -q)

Create a JDG app and scale to a cluster size of 3

# Using a pre-defined template as it starts a set of resources 
# http-session-cache is the cache that will hold the external EAP session information
oc new-app --template=datagrid65-basic -p CACHE_NAMES=http-session-cache

# Scale to cluster size of three after ensuring the first single pod came up correctly
oc scale --replicas=3 dc datagrid-app

Verify clustering between JDG pods

  1. Within the HTTP Session Externalization into JDG project go to Browse → Pods
  2. Click on any of the JDG pods named like datagrid-app-x-xxxxx in Running status
  3. Staying under the Details tab, click on the Open Java Console
  4. In the JMX tree, choose jboss.infinispan → CacheManager → clustered → CacheManager and verify the Cluster size, it should be 3

Create a EAP app and scale to a cluster size of 2

oc new-app --template=eap64-basic-s2i \
   -p SOURCE_REPOSITORY_URL=https://github.com/vchintal/http-session-counter-openshift.git,SOURCE_REPOSITORY_REF=,CONTEXT_DIR=/

# Scale to cluster size of two after ensuring the first single pod came up correctly
oc scale --replicas=2 dc eap-app

Verify clustering between EAP pods

Using the above images as a reference, follow the steps below to ensure EAP pods are clustering

  1. Within the HTTP Session Externalization into JDG project go to Browse → Pods
  2. Click on any of the EAP pods named like eap-app-x-xxxxx in Running status
  3. Staying under the Details tab, click on the Open Java Console
  4. In the JMX tree, choose jboss.infinispan → CacheManager → web → CacheManager and verify the Cluster size, it should be 2

Test the HTTP session externalization

For the testing we will use the Firefox browser and its Developer Tools, especially the Network component.

  1. Run oc get routes and find out the service url for EAP pods
  2. Access the service url and append the context path /http-session-counter, the counter is set to 1 in the session
  3. Enable the Network tool of the Developer Tools and then re-run/refresh the same url, the counter is now set to 2 as shown in the image below (Cluster IP and Port 8080 of the service is used in this example)
  4. Now capture the cURL command for the network call by right-clicking on the blue highlighted entry below and choosing Copy as cURL
  5. Paste the cURL command in terminal and run it couple of times
  6. Run the following commands in the sequence from the command line as shown
    • oc project http-session-externalization
    • oc scale --replicas=0 dc eap-app : This will shutdown all the EAP pods
    • oc scale --replicas=2 dc eap-app : This will create two fresh new EAP pods
  7. Now go back to the window/terminal where you ran the step # 5 and re-run the same curl command. Do you see that the counter got incremented and a new pod-name is displayed ?

If the answer to the question is a resounding Yes then we have successfully completed persisting EAP HTTP sessions into a remote JDG cluster. Also, after hitting the application URL by various means to ensure enough sessions have been created, you can verify the total entry count by navigating to a JDG pod as before and using the Java Console to verify the number of entries in the cache as shown below

References

  1. EAP xPaaS Image Documentation
  2. JDG xPaaS Image Documentation
  3. HTTP Session Externalization on EAP
comments powered by Disqus