Hibernate 5 - Batch processing example

Posted on June 2, 2018


This post shows you how to perform the INSERT and UPDATE batch operations in Java using the Hibernate ORM framework.

Hibernate stores the new inserted objects in the second-level cache. Inserting more than one million objects in single iteration can cause the OutOfMemoryException. To avoid this exception we can enable the batching in hibernate.

Hibernate leverage the JDBC batching which allows us to send multiple statements to database server in one call.

In hibernate, batch size can be set using the property hibernate.jdbc.batch_size  to an integer between 10 and 50.

Hibernate disables the INSERT batching for entities using the IDENTITY generator. So In this example, we will use the JPA @TableGenerator annotation to generate the unique key for entity.

Tools and technologies used for this application are - 

  • Hibernate ORM 5.2.17.Final
  • Log4j 2.11.0
  • JAXB API 2.3.0
  • JavaSE 9
  • MySQL Server 5.7.12
  • Eclipse Oxygen.1
  • Maven 3.3.9

Jar dependencies

Add the following jar dependencies in pom.xml file of your maven project.

<dependencies>
   <dependency>
      <groupId>org.hibernate</groupId>
      <artifactId>hibernate-core</artifactId>
      <version>5.2.17.Final</version>
   </dependency>
   <dependency>
      <groupId>mysql</groupId>
      <artifactId>mysql-connector-java</artifactId>
      <version>6.0.6</version>
   </dependency>
   <dependency>
      <groupId>javax.xml.bind</groupId>
      <artifactId>jaxb-api</artifactId>
      <version>2.3.0</version>
   </dependency>

   <!--Log4j2 API -->
   <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-core</artifactId>
      <version>2.11.0</version>
   </dependency>
   <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-api</artifactId>
      <version>2.11.0</version>
   </dependency>
</dependencies>

 Entity class

Create a simple @Entity class as follows.

Employee.java

package com.boraji.tutorial.hibernate.entity;

import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.TableGenerator;

@Entity
public class Employee {
   
   @Id
   @TableGenerator(name = "EMP_SEQ")
   @GeneratedValue(strategy = GenerationType.TABLE, generator = "EMP_SEQ")
   private Long id;

   private String name;
   private String emial;
 
  //Getter and Setter methods
}

 

Configuring batching in Hibernate

In order to configure the batching in hibernate application, you need to specify the following property in configuration class -

hibernate.jdbc.batch_size = <Positive Integer number>

Setting zero or negative number will disable the batching.

Here is the complete example of hibernate configuration class to bootstrap application.

HibernateUtil.java

package com.boraji.tutorial.hibernate;

import java.util.HashMap;
import java.util.Map;

import org.hibernate.SessionFactory;
import org.hibernate.boot.Metadata;
import org.hibernate.boot.MetadataSources;
import org.hibernate.boot.registry.StandardServiceRegistry;
import org.hibernate.boot.registry.StandardServiceRegistryBuilder;
import org.hibernate.cfg.Environment;

import com.boraji.tutorial.hibernate.entity.Employee;

public class HibernateUtil {
   private static StandardServiceRegistry registry;
   private static SessionFactory sessionFactory;

   public static SessionFactory getSessionFactory() {
      if (sessionFactory == null) {
         try {
            StandardServiceRegistryBuilder registryBuilder = new StandardServiceRegistryBuilder();

            //Configuration properties
            Map<String, Object> settings = new HashMap<>();
            settings.put(Environment.DRIVER, "com.mysql.cj.jdbc.Driver");
            settings.put(Environment.URL, "jdbc:mysql://localhost:3306/BORAJI?useSSL=false");
            settings.put(Environment.USER, "root");
            settings.put(Environment.PASS, "admin");
           
            settings.put(Environment.HBM2DDL_AUTO, "update");
            //Set JDBC batch size
            settings.put(Environment.STATEMENT_BATCH_SIZE, 50);

            registryBuilder.applySettings(settings);
            registry = registryBuilder.build();
            
            MetadataSources sources = new MetadataSources(registry);
            sources.addAnnotatedClass(Employee.class);
            Metadata metadata = sources.getMetadataBuilder().build();
            
            sessionFactory = metadata.getSessionFactoryBuilder().build();
         } catch (Exception e) {
            if (registry != null) {
               StandardServiceRegistryBuilder.destroy(registry);
            }
            e.printStackTrace();
         }
      }
      return sessionFactory;
   }

   public static void shutdown() {
      if (registry != null) {
         StandardServiceRegistryBuilder.destroy(registry);
      }
   }
}

Logging

In order to check whether batch insert or update operations are working or not, you can add the BatchingBatch in log4j2.xml file as follows.

log4j2.xml

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
   <Appenders>
      <!-- Console Appender -->
      <Console name="Console" target="SYSTEM_OUT">
         <PatternLayout pattern="%d{yyyy-MMM-dd HH:mm:ss a} [%t] %-5level %logger{36} - %msg%n" />
      </Console>
   </Appenders>
   <Loggers>
      <!-- Log everything in hibernate -->
      <Logger name="org.hibernate.engine.jdbc.batch.internal.BatchingBatch" level="debug" additivity="false">
         <AppenderRef ref="Console" />
      </Logger>
      <Root level="error">
         <AppenderRef ref="Console" />
      </Root>
   </Loggers>
</Configuration>

Hibernate batch INSERT example

To perform the INSERT batch operation, you need to invoke the Session.flush() and Session.clear() methods regularly, to controller the size of second-level cache as follows.

InsertBatchExample.java

package com.boraji.tutorial.hibernate;

import org.hibernate.Session;
import org.hibernate.Transaction;

import com.boraji.tutorial.hibernate.entity.Employee;

public class InsertBatchExample {
   public static void main(String[] args) {
      Session session = null;
      Transaction transaction = null;
      int batchSize = 50;
      try {
         session = HibernateUtil.getSessionFactory().openSession();
         transaction = session.beginTransaction();

         for (long i = 1; i <= 100000; i++) {
            Employee employee = new Employee();
            employee.setName("Employee " + i);
            employee.setEmial("[email protected]");
            session.save(employee);
            if (i > 0 && i % batchSize == 0) {
               session.flush();
               session.clear();
            }
         }
         transaction.commit();
      } catch (Exception e) {
         e.printStackTrace();
      } finally {
         if (session != null) {
            session.close();
         }
      }
      
      HibernateUtil.shutdown();
   }
}

On executing the above program, you will get the output as follows -

2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
........
....

 

Hibernate batch UPDATE example

To perform the UPDATE batch operation, you can use the ScrollableResults to fetch the bulk objects then  invoke the Session.flush() and Session.clear() methods regularly as follows.

UpdateBatchExample.java

package com.boraji.tutorial.hibernate;

import org.hibernate.CacheMode;
import org.hibernate.ScrollMode;
import org.hibernate.ScrollableResults;
import org.hibernate.Session;
import org.hibernate.Transaction;

import com.boraji.tutorial.hibernate.entity.Employee;

public class UpdateBatchExample {
   public static void main(String[] args) {
      Session session = null;
      Transaction transaction = null;
      ScrollableResults scrollableResults = null;
      int batchSize = 50;
      try {
         session = HibernateUtil.getSessionFactory().openSession();
         transaction = session.beginTransaction();

         scrollableResults=session.createQuery("from Employee")
               .setCacheMode(CacheMode.IGNORE)
               .scroll(ScrollMode.FORWARD_ONLY);
         
         int count=0;
         while (scrollableResults.next()) {
            Employee employee = (Employee) scrollableResults.get(0);
            employee.setEmial("[email protected]"); // Change property
            if (++count % batchSize == 0) {
               session.flush();
               session.clear();
            }
         }
         
         transaction.commit();
      } catch (Exception e) {
         e.printStackTrace();
      } finally {
         if (scrollableResults != null) {
            scrollableResults.close();
         }
         if (session != null) {
            session.close();
         }
      }
      
      HibernateUtil.shutdown();
   }
}

On executing the above program, you will get the output as follows -

2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
2018-Jun-02 17:00:52 PM [main] DEBUG org.hibernate.engine.jdbc.batch.internal.BatchingBatch - Executing batch size: 50
........
....
Download Sources