This blog will show a simple use of Spring-batch framework. Spring-batch is a spring framework made for processing batch jobs. It’s very stable and supports a large number of registries.

This job will read a plain text file (CSV) and write the information to a Database.

The Database (DB) we are using is H2, because of its simplicity. Of course, it could use any other DB, or even no DB at all (as you will see further down).

In this blog, we used an XML configuration, but it will work using annotations too (this may be a subject for a future blog).

Maven Configuration

The first task is to configure maven dependencies in pom.xml for the spring core and spring-batch project:

Define the version that will be used for each one:

<properties>
<jdk>1.7</jdk>
<springbatch-version>2.2.0.RELEASE</springbatch-version>
<spring-version>3.2.2.RELEASE</spring-version>
</properties>

Add dependencies:

<!-- spring Core -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>${spring-version}</version>
</dependency>

<!-- spring-batch dependencies -->
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-infrastructure</artifactId>
<version>${springbatch-version}</version>
</dependency>
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-core</artifactId>
<version>${springbatch-version}</version>
</dependency>

<!-- spring database dependencies -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jdbc</artifactId>
<version>${spring-version}</version>
</dependency>

Spring configuration

Datasource configuration.

Now, in order to use H2, you will need to add the maven dependency:

<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.4.187</version>
</dependency>

Create a file datasource-config.xml where you will configure the datasources for the DB connection:

<!-- configure H2 database connection -->
<bean id="dataSourceH2"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.h2.Driver" />
<property name="url" value="jdbc:h2:~/test" />
</bean>

<bean id="transactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />

Create a file spring-config.xml where you will configure all the beans.

Configure jobLauncher:

There are 2 options in order to configure it: in Memory:

<!-- store jobs in memory -->
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
<property name="transactionManager" ref="transactionManager" />
</bean>

<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>

Or using DB: In this case, it’s needed to change the “jobRepository” configuration:

<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
<property name="dataSource" ref="dataSourceH2" />
<property name="transactionManager" ref="transactionManager" />
<property name="databaseType" value="h2" />
</bean>

If you are going to use a DB for the springbatch job, some tables will need to be created to store it. To create these tables when the DB starts up, just add:

<jdbc:initialize-database data-source="dataSourceH2">
<jdbc:script location="org/springframework/batch/core/schema-drop-h2.sql" />
<jdbc:script location="org/springframework/batch/core/schema-h2.sql" />
</jdbc:initialize-database>

Configure the job:

This job will read a CSV file and will store it in the DB.

A spring-batch job has a reader and a writer.

The reader is the bean which will read the file. In order to do this, it will convert each registry (or item) from the file into a POJO. And the writer is, of course, going to write the information into the DB.

In this case, we use only 1 step, but you could add more steps, before and after reading the file. You can also pass data through them (I will demonstrate this in a future blog).

The configuration steps will be:

  • Create the POJO that represents an item from the file.
  • Configure the reader.
  • Configure the writer.
  • Configure the job.

Create the POJO:

public class PersonBean {

private int id;
private String name;
private String address;

public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getAddress() {
return address;
}
public void setAddress(String address) {
this.address = address;
}

}

Create a file springbatch-config.xml where you will configure the job:

Define the POJO:

<bean id="personBean" class="com.aaj.springbatchpoc.PersonBean" scope="prototype" />

Configure the reader:

The reader will read each file from the CSV and convert it to a POJO. So, we need to define it and configure it in spring:

<bean id="csvFileReader" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="resource" value="classpath:/sample.csv" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean class=
"org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="names" value="id, name, address" />
</bean>
</property>
<property name="fieldSetMapper">
<bean class=
"org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
<property name="prototypeBeanName" value="personBean" />
</bean>
</property>
</bean>
</property>
</bean>

Configure the writer:

<bean id="itemWriter" class="org.springframework.batch.item.database.JdbcBatchItemWriter">
<property name="dataSource" ref="dataSourceH2" />
<property name="sql">
<value>
<![CDATA[
insert into persons(ID, NAME, ADDRESS)
values (:id, :name, :address)]]>
</value>
</property>
<property name="itemSqlParameterSourceProvider">
<bean
class="org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider" />
</property>
</bean>

Now configure the job. I have added an “extra” step 1, just to show that a job could have more than one step, not only the step that reads and writes. You can add steps 3, 4, etc.

<batch:job id="SampleJob">
<batch:step id="step1">
<batch:tasklet>
<batch:chunk reader="csvFileReader" writer="itemWriter" commit-interval="10">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>

Now we need to create the class that will start the application and execute the job:

public class Main {

private static final String className = Main.class.getSimpleName();

public static void main(String[] args) {

String[] configuration = { "spring-config.xml", "springbatch-config.xml", "datasource-config.xml"};

ApplicationContext context = new ClassPathXmlApplicationContext(configuration); JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
Job sampleJob = (Job) context.getBean("SampleJob");

try {
JobExecution jobExecution = jobLauncher.run(sampleJob, new JobParameters());

} catch (Exception e) {
e.printStackTrace();
}
}
}

Now the only that is missing is the CSV file. We created one with sample data:

01, Alejandro Robledo, address 1
02, Name 2, Address 2
03, Name 3, Address 3

Testing the job – Junit

The job can be tested completed, or partially (in steps). In this case I will use Junit in order to demonstrate it.

First we have to add the dependencies for the Spring-batch test and Junit:

<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-test</artifactId>
<version>2.2.0.RELEASE</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
</dependency>

Also it’s necessary to add a JobLauncher for the test in the spring-config.xml file:

<bean class="org.springframework.batch.test.JobLauncherTestUtils"/>

In order to keep the test from the source code, we are going to create a new folder “test” and set it to use as a “src folder” (in eclipse, right click->Build path -> Uses as Source folder).

Then we will create the test class inside this folder:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = {
"classpath:database.xml",
"classpath:context.xml",
"classpath:job-report.xml" })
public class AppTest {

@Autowired
private JobLauncherTestUtils jobLauncherTestUtils;

@Test
public void launchJobTest() throws Exception {
JobExecution jobExecution = jobLauncherTestUtils.launchJob();
assertEquals("testing complete Job", BatchStatus.COMPLETED, jobExecution.getStatus());
}

@Test
public void executeStep2Test() throws Exception{
JobExecution jobExecution = jobLauncherTestUtils.launchStep("step1");
assertEquals("Testing step1.", BatchStatus.COMPLETED, jobExecution.getStatus());
}

Now, in order to maven to run the tests, we need to define where they are located and which resources they need to run, in pom.xml:

<testSourceDirectory>test</testSourceDirectory>
<testResources>
<testResource>
<directory>resources</directory>
</testResource>
</testResources>

Then we could run the maven command to see if the tests run and if they passed, executing (in console): mvn clean test.

Hopefully, this was a helpful example. I will be writing more in the near future.