Spring Boot application

These are two applications that are connected but are source-controlled, developed and deployed separately.

The Spring app uses a h2 database, the main application sets up some initial data, using a model and a repository. We also have a logging endpoint to collect client-side Angular logs.

Create a normal Spring Boot app with the Initializer web tool or in STS with the following dependencies:

• spring-boot-starter-data-jpa
• spring-boot-starter-data-rest
• spring-boot-starter-web
• spring-boot-starter-test
• h2

And optionally:

• spring-boot-devtools
• lombok

The Java code goes under src/main/java in the package com.backtojavaland and contains only 4 files.

• Application.java
• User.java
• UserRepository.java
• Log.java

//Application.java
package com.backtojavaland;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class Application implements CommandLineRunner {

	@Autowired
	UserRepository userRepository;
	
	public static void main(String[] args) {
		SpringApplication.run(Application.class, args);
	}
	
	//bootstrap database at start of application
	public void run(String... strings) throws Exception {
		userRepository.save(new User("Ada","Lovelace"));
		userRepository.save(new User("Grace","Hopper"));
	}
}

Here we’re using CommandLineRunner to add a few lines to the database. An alternative way would be to create schema.sql and data.sql files. I’d do this if using a MySQL database. See https://docs.spring.io/spring-boot/docs/current/reference/html/howto-database-initialization.html.

//User.java
package com.backtojavaland;

import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;

@Entity
public class User {

	@Id
	@GeneratedValue(strategy = GenerationType.AUTO)
	private Long id;
	private String name;
	private String surname;
	
	public User() {} 
	
	public User(String name, String surname) {
		this.name = name;
		this.surname = surname;
	}
	
	//getters and setters
}

@Entity, a simple User domain object, we need no-arg constructor for JPA. 

//UserRepository.java
package com.backtojavaland;

import org.springframework.data.repository.PagingAndSortingRepository;
import org.springframework.data.rest.core.annotation.RepositoryRestResource;

@RepositoryRestResource(collectionResourceRel = "user", path="user")
public interface UserRepository extends PagingAndSortingRepository<User, Long>{

}

This is just magic! We use the PagingAndSortingRepository interface to handle CRUD operations for us. The @RepositoryRestResource annotation allows us to optionally customize the REST endpoint. It defaults to the name of the domain object, and we just reiterate that here.

Now lets add a simple logging endpoint that just prints things out for now to the console:

//Log.java
package com.backtojavaland;

import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class Log {

	@PostMapping("/logs")
	public void getLog(@RequestBody String request) {
		System.out.println(request);
	}
}

#application.properties

# make sure all REST endpoints exposed by spring boot app is under /api url
server.servlet.context-path=/api

# H2 Web Console (H2ConsoleProperties)
spring.h2.console.enabled=false # Whether to enable the console.
spring.h2.console.path=/h2-console # Path at which the console is available.
spring.h2.console.settings.trace=false # Whether to enable trace output.
spring.h2.console.settings.web-allow-others=false # Whether to enable remote access.

Build and runs on http://localhost:8080 and has two endpoints:

• /users – has ?page, ?size and ?sort options
• /profile – has metadata for application

Spring Data REST automatically follows the principles of HATEOAS too, so the content-type returned is application/hal+json.

As we already have some data in the database, we can search for them:

http://localhost:8080/users/search/findByName?name=test

There are lots of methods provided by the PagingAndSortingRepository that we can use ‘out-of-the-box’. See https://www.baeldung.com/spring-data-repositories for a quick overview.

Angular application

Create an angular application called angular-frontend.

Add a new file called proxy.conf.json:

{
	"/api":{
		"target" : "http://localhost:8080",
		"secure" : false	
	}
}

Modify package.json to load up the proxy-config file.

You’ll need to use npm start to pick this file up in future!

"start": "ng serve --proxy-config proxy.conf.json",

Now add the application code.

//app.module.ts
import { BrowserModule } from '@angular/platform-browser';
import { NgModule } from '@angular/core';
import { AppComponent } from './app.component';
import { LoggerModule, NgxLoggerLevel } from 'ngx-logger';

@NgModule({
  declarations: [
    AppComponent
  ],
  imports: [
    BrowserModule,
    LoggerModule.forRoot({serverLoggingUrl: '/api/logs', level: NgxLoggerLevel.DEBUG, serverLogLevel: NgxLoggerLevel.ERROR}),
  ],
  providers: [],
  bootstrap: [AppComponent]
})
export class AppModule { }

Notice that we’ve added a logger and set it up to log to the server. We have a rest endpoint that just prints this out on the console for now.

//app.component.ts
import { Component } from '@angular/core';
import { NGXLogger } from 'ngx-logger';
import { Injectable } from '@angular/core';
import { HttpClient } from '@angular/common/http';
import { Observable } from 'rxjs';
import { map } from 'rxjs/operators';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css'],
  providers: [NGXLogger]
})
export class AppComponent {
  title = 'frontend';
  users;

  constructor(public http: HttpClient, private logger: NGXLogger){
  	this.logger.debug('Your log message goes here');
    this.logger.debug('Multiple', 'Argument', 'support');
  	this.getUser().subscribe(
  		res => this.users = res._embedded.user,
  		error => console.log(error)
  	);
  	
  }

  getUser(): Observable<any>{
  	return this.http.get('http://localhost:4200/api/user').pipe(
		map(respnse => <string[]> respnse)
	);
        
  }
}

Notice I’ve put a few logging comments in here. Angular logs to the client by default.

And then to see the data, a html file:

<h1>
	Hi! {{ title }}
</h1>

<ol>
	<li *ngFor="let user of users">
		{{user.name}}
		{{user.surname}}
	</li>
</ol>

To run the applications they need to be deployed separately. The Spring Application is launched on http://localhost:8080 and the Angular application on 
http://localhost:4200. The angular app talks to the spring boot application via REST and gets the data.

I’m really happy with this example, it’s a great starting point. Now, let’s make it pretty by adding Bootstrap.

tl;dr: no pretty graphs 😦 

I’ve never heard of this before, it’s a DevOps kind of thing. Jolokia will expose the Actuator JMX beans over HTTP at the Jolokia endpoint. This is so that the metrics exposed by Actuator can be sent to other services to more easily make sense of them and to make pretty graphs.

The idea in the tutorial is to send the data from Spring Boot Actuator via Telegraf  to InfluxDB. Then Grafana pulls the data to make pretty Graphs.

• InfluxDB – time series database
• Telegraf – server for collecting and writing metrics
• Grafana – server used to visualize time-series data

This seems a bit nuts to me, how many tools do we need ffs. Anyway, lets go with the flow.

Add jolokia to the pom.xml file so Spring Boot can auto-configure the jolokia end-point.

<dependency>
<groupId>org.jolokia</groupId>
<artifactId>jolokia-core</artifactId>
</dependency>

version: '2.1'

[tags]
dc = "local-1"

[agent]
interval = "5s"

[[inputs.jolokia]]
context = "/jolokia"

[[inputs.jolokia.servers]]
name = "jolokia-server"
host = "section-2-3-service"
port = "8080"

[[inputs.jolokia.metrics]]
name = "metrics"
mbean = "org.springframework.boot:name=metricsEndpoint,type=Endpoint"
attribute = "Data"

[outputs]
[outputs.influxdb]
url = "http://influxdb:8086"
database = "metrics"
precision = "s"

#[[inputs.jolokia]]
# context = "/jolokia/"

#[[inputs.jolokia.servers]]
# name = "jolokia-server"
# host = "section-2-3-service"
# port = "8080"

#[[inputs.jolokia.metrics]]
# name = "metrics"
# mbean = "org.springframework.boot:name=metricsEndpoint,type=Endpoint"
# attribute = "Data"

# https://github.com/influxdata/telegraf/blob/master/etc/telegraf.conf
# # Read JMX metrics from a Jolokia REST agent endpoint
[[inputs.jolokia2_agent]]
# # default_tag_prefix = ""
# # default_field_prefix = ""
# # default_field_separator = "."
#
# # Add agents URLs to query
urls = ["http://localhost:8080/application/jolokia"]
# # username = ""
# # password = ""
# # response_timeout = "5s"
#
# ## Optional TLS config
# # tls_ca = "/var/private/ca.pem"
# # tls_cert = "/var/private/client.pem"
# # tls_key = "/var/private/client-key.pem"
# # insecure_skip_verify = false
#
# ## Add metrics to read
[[inputs.jolokia2_agent.metric]]
name = "metrics"
mbean = "org.springframework.boot:name=metricsEndpoint,type=Endpoint"
paths = ["Data"]

I changed the urls to urls = [“http://localhost:8080/application/jolokia”%5D because that was what was being mapped at start-up. I removed the old target folder, built the application again and re-ran the docker-compose up stuff. Now a different error:

telegraf | 2018-11-07T22:06:50Z E! Error in plugin [inputs.jolokia2_agent]: Unable to gather metrics for http://localhost:8080/application/jolokia: Post http://localhost:8080/application/jolokia/read: dial tcp 127.0.0.1:8080:
connect: connection refused

Urgh! However, looking at http://localhost:8080/application/jolokia in the browser, I got some JSON back!!!! I formatted it below to have a good look at it.

{"request":{"type":"version"},
 "value":{"agent":"1.3.7",
          "protocol":"7.2",
          "config":{"agentId":"172.18.0.3-6-2e77b8cf-servlet",
                    "agentType":"servlet"},
          "info":{"product":"tomcat",
                  "vendor":"Apache",
                  "version":"8.5.16"}
         },
 "timestamp":1541628160,
 "status":200
}

When I navigate to http://localhost:8080/application/jolokia/read I get:

{"stacktrace":"java.lang.IllegalArgumentException: 
Invalid arguments in pathinfo read for command READ\n\t
at org.jolokia.request.JmxRequestFactory.createGetRequest
bla bla bla

Found this from stackoverflow: https://stackoverflow.com/questions/38838526/how-to-use-jolokia-to-retrive-mbean-for-a-class

In general, you can get a list of all your available mbeans like this:

http://yourserver/jolokia/list

You should end up with a large json document that contains everything you might want to fetch. You will see things like

"foo.bar.Log4j": {
  "name=foo,type=MyLogger": {
    "desc": ...
    "attr": {
        ...
}}}

You can now get the attributes using something like this:

http://yourserver/jolokia/read/foo.bar.Log4j:type=name=foo,type=MyLogger

In addition to type and name, you may see other fields as well, for example context or id. This a:b key is the Java ObjectName for your mbean.

Cool, I did it and got back loads of stuff. I searched for metricsEndpoint from the tutorial telegraf.conf:

#[[inputs.jolokia.metrics]]
# name = "metrics"
# mbean = "org.springframework.boot:name=metricsEndpoint,type=Endpoint"
# attribute = "Data"

and found it. Used JSONLint online to prettify it:

"class": "org.springframework.boot.actuate.endpoint.jmx.AuditEventsJmxEndpoint", "desc": ""
}, "name=metricsEndpoint,type=Endpoint": {
	"op": {
		"isSensitive": {
			"args": [],
			"ret": "boolean",
			"desc": "Indicates whether the underlying endpoint exposes sensitive information"
		},
		"getEndpointClass": {
			"args": [],
			"ret": "java.lang.String",
			"desc": "Returns the class of the underlying endpoint"
		},
		"getData": {
			"args": [],
			"ret": "java.lang.Object",
			"desc": "Invoke the underlying endpoint"
		}
	},
	"attr": {
		"EndpointClass": {
			"rw": false,
			"type": "java.lang.String",
			"desc": "Returns the class of the underlying endpoint"
		},
		"Sensitive": {
			"rw": false,
			"type": "boolean",
			"desc": "Indicates whether the underlying endpoint exposes sensitive information"
		},
		"Data": {
			"rw": false,
			"type": "java.lang.Object",
			"desc": "Invoke the underlying endpoint"
		}
	},

Look, there’s an attribute for Data like before.

So, like the stackoverflow advice, I should be able to go

http://localhost:8080/application/jolokia/read/org.springframework.boot.actuate.endpoint.jmx.AuditEventsJmxEndpoint:name=metricsEndpoint,type=Endpoint

Got stuff 😀 but it seems that the name was wrong.

{“request”:{“mbean”:”org.springframework.boot.actuate.endpoint.jmx.AuditEventsJmxEndpoint:name=metricsEndpoint,type=Endpoint”,”type”:”read”},”stacktrace”:”javax.management.InstanceNotFoundException:

I just tried typing in this from the mbean part:

http://localhost:8080/application/jolokia/read/org.springframework.boot:name=metricsEndpoint,type=Endpoint

and I got better stuff 😀

{
	"request": {
		"mbean": "org.springframework.boot:name=metricsEndpoint,type=Endpoint",
		"type": "read"
	},
	"value": {
		"EndpointClass": "org.springframework.boot.actuate.endpoint.MetricsEndpoint",
		"Sensitive": false,
		"Data": {
			"heap.committed": 204800,
			"heap.used": 144214,
			"classes": 7670,
			"httpsessions.max": -1,
			"systemload.average": 0.01,
			"mem.free": 60585,
			"processors": 2,
			"gc.ps_marksweep.count": 2,
			"threads.peak": 23,
			"mem": 263926,
			"counter.status.200.application.root": 1,
			"counter.status.404.application.metrics.name": 1,
			"nonheap.used": 59126,
			"gauge.response.application.jolokia.star-star": 14.0,
			"heap": 451584,
			"gc.ps_marksweep.time": 157,
			"nonheap.init": 2496,
			"counter.status.200.application.metrics": 1,
			"threads": 21,
			"heap.init": 32768,
			"gauge.response.application.metrics": 29.0,
			"threads.totalStarted": 26,
			"gc.ps_scavenge.count": 12,
			"uptime": 1952627,
			"nonheap": 0,
			"gauge.response.application.root": 209.0,
			"instance.uptime": 1939639,
			"httpsessions.active": 0,
			"nonheap.committed": 60800,
			"threads.daemon": 19,
			"classes.unloaded": 0,
			"counter.status.200.application.jolokia.star-star": 8,
			"classes.loaded": 7670,
			"gc.ps_scavenge.time": 159,
			"gauge.response.application.metrics.name": 83.0
		}
	},
	"timestamp": 1541632046,
	"status": 200
}

This is brilliant, but why can’t the application get this data???

I’m next going to focus on why it’s doing a Post and not a Get. Looking at https://jolokia.org/reference/html/protocol.html

Jolokia requests can be sent in two ways: Either as a HTTP GET request, in which case the request parameters are encoded completely in the URL. Or as a POST request where the request is put into a JSON payload in the HTTP request’s body. GET based requests are mostly suitable for simple use cases and for testing the agent via a browser. The focus here is on simplicity. POST based requests uses a JSON representation of the request within the HTTP body. They are more appropriate for complex requests and provide some additional features (e.g. bulk requests are only possible with POST).

The response returned by the agent uses always JSON for its data representation. It has the same format regardless whether GET or POST requests are used.

I used Postman to do the browser GET call again and it worked. Then I tried to do the POST. After playing around a bit (I forgot to use ‘:’ instead of ‘=’) I put the following JSON in the Body:

{
“type” : “read”,
“name” : “metrics”,
“mbean” : “org.springframework.boot:name=metricsEndpoint,type=Endpoint”
}

It couldn’t find it!!!! Yey!!!

So, I re-checked the telegraf.conf file and noticed something… my mbean definition was different:

mbean = “org.springframework.boot.actuate.metrics:name=metricsEndpoint,type=Endpoint”

This was left over from something else I tried. I renamed it to the tutorial example again, and removed the Data bit.

# ## Add metrics to read
[[inputs.jolokia2_agent.metric]]
name = "metrics"
mbean = "org.springframework.boot:name=metricsEndpoint,type=Endpoint"

Re-built project, docker-compose up and… same error.

Ok, changed telegraf.conf again:

# ## Add metrics to read
[[inputs.jolokia2_agent.metric]]
type = "read"
name = "metrics"
mbean = "org.springframework.boot:name=metricsEndpoint,type=Endpoint"

It didn’t like the type stuff in the telegraf.conf file:

telegraf | 2018/11/08 00:05:57 E! Error parsing /etc/telegraf/telegraf.conf, line 81: field corresponding to `type’ is not defined in `*jolokia2.MetricConfig’

Removed type and added paths for the Data bit I forgot about… maybe that will help?

# ## Add metrics to read
[[inputs.jolokia2_agent.metric]]
name = "metrics"
mbean = "org.springframework.boot:name=metricsEndpoint,type=Endpoint"
paths = ["Data"]

Nope. Added paths in Postman and it returned just the same. Hmmmm… it’s not that.

Google: https://github.com/influxdata/telegraf/issues/4371

Since you are running in docker you will need to make sure Telegraf can reach the container you application is in. This depends a bit on how you are starting your containers, in some simple cases you may just need to start the Telegraf container with something like --net=container:yourappcontainer. I also suggest using the docker exec command to run a shell in your Telegraf container for debugging.

Honestly, this makes no sense to me. I was hoping for a tutorial to just click through and learn something. This has been a  minefield and a good look into the DevOps world… urgh! Scratching around on Google for information was impossible. I suppose I have an appreciation of more technologies, just wish i could have had a pretty graph 😦

I’ve decided to create a proper front-end, so will be taking this Safari course. I also took a Live Lesson, but have yet to properly look at it. Unfortunately, you cannot speed up the videos of the live lessons, and they’re 6 hours long 😦 I like to listen to everything on at least 1.5x, mostly 2x! I may look at the slides later and see if anything is new. So, today I’ll be looking at the above course by two different people. It seems to have been mashed together, so I wonder how contiguous it is…