Datum Capture Part 1

SolarNode Datum Capture Plug-in Example

This guide explains how to create a SolarNode plug-in to capture data from a fictional solar inverter, using the Eclipse IDE. This is the first part in a series of lessons. This part will guide you through:

Creating a SolarNode plug-in project in Eclipse
Implementing code for capturing power data (e.g. Wh generated) from a fake Foobar brand solar inverter
Creating an associated JUnit unit test project with some unit tests

The code for this example is available as well.

Eclipse Setup

If you haven't already set up your SolarNetwork development environment, either go through the SolarNode Developement Guide or the Developer VM guide first, and then return here.

Create new Plug-in project for Foobar Power

The SolarNode runtime is based on OSGi and when you build a SolarNode plug-in you're actually building an OSGi bundle, which can be thought of as a normal Java JAR archive with some additional metadata stored in the MANIFEST.MF file included in the JAR.

Eclipse refers to OSGi bundles as "plug-ins" and its OSGi development tools are collectively known as the Plug-in Development Environment, or PDE. Create a new Plug-in project by selecting File > New > Project.... From the dialog window that appears, select Plug-in Project and click Next.

In the next screen, fill in some details for the new project. We'll be following SolarNode naming conventions to create a bundle that collects power generation data from a fictional Foobar solar inverter. The project name will mirror our OSGi bundle name: net.solarnetwork.node.example.datum-capture. Fill in the following details:

Project name - net.solarnetwork.node.example.datum-capture
Output folder - set to build/eclipse so we can support standard SolarNode Ant-based builds later on.
Target Platform - set to a standard OSGi framework

Click the Next > button and on the next screen, fill in the OSGi bundle information:

ID - the same as the project name
Version - can start as 1.0.0
Name - something descriptive, e.g. Example Datum Capture
Vendor - an appropriate vendor, or SolarNetwork for a SolarNetwork-sponsored plug-in
Execution Environment - set to JavaSE-1.6

That's all you need to configure to get started, so click Finish now.

SolarNode datum

SolarNode collects samples of data from sensors, meters, etc. and refers to each sample as a datum. SolarNode provides a basic API all datum are required to implement, net.solarnetwork.node.domain.Datum, which looks like this:

public interface Datum {

	/**
	 * Get the date this object was created, which is often equal to either the
	 * date it was persisted or the date the associated data in this object was
	 * captured.
	 *
	 * @return the created date
	 */
	public Date getCreated();

	/**
	 * Get a unique source ID for this datum.
	 *
	 * <p>
	 * A single datum type may collect data from many different sources.
	 * </p>
	 *
	 * @return the source ID
	 */
	public String getSourceId();

	/**
	 * Get the date this object was uploaded to SolarNet.
	 *
	 * @return the upload date
	 */
	public Date getUploaded();

}

We want to collect samples of power generation our Foobar inverter, but notice that there are no properties specific to power generation on the Datum interface, such as instantaneous watts or an accumulated watt-hour reading value. SolarNode provides an extension of that interface called net.solarnetwork.node.domain.EnergyDatum that does include those properties, however. Here's a simplified view of that API:

public interface EnergyDatum extends Datum {

	/**
	 * Get a watt-hour reading. Generally this is an accumulating value and
	 * represents the overall energy used or produced since some reference date.
	 * Often the reference date if fixed, but it could also shift, for example
	 * shift to the last time a reading was taken.
	 *
	 * @return the watt hour reading, or <em>null</em> if not available
	 */
	public Long getWattHourReading();

	/**
	 * Get the instantaneous watts.
	 *
	 * @return watts, or <em>null</em> if not available
	 */
	public Integer getWatts();

	// more properties here...

}

SolarNode then provides a class that implements this API, along with mutable property setters, called net.solarnetwork.node.domain.GeneralNodePVEnergyDatum that looks like this (again, simplified for this guide):

public class GeneralNodePVEnergyDatum implements Datum {

	// the basic Datum API setters (getters not shown)...

	public void setCreated(Date created) { }
	public void setSourceId(String sourceId) { }
	public void setUploaded(Date uploaded) { }

	// followed by EnergyDatum specific properties...

	public void setWatts(Integer watts) { }            // instantaneous generation

	public void setWattHourReading(Long whReading) { } // accumulating Wh reading

	// more properties here...
}

Now we know we want to read samples from our Foobar inverter and translate the samples into GeneralNodePVEnergyDatum instances. Next, we look at the API that handles producing those Datum samples.

SolarNode datum data source

SolarNode defines two APIs for classes that can sample data and produce Datum instances, referred to as data sources. Those APIs are defined by these interfaces:

net.solarnetwork.node.DatumDataSource
net.solarnetwork.node.MultiDatumDataSource

These APIs are very simple: they expose the type of Datum they can produce and they return new datum instances of that type when asked. The general idea is that SolarNode will periodically query the data source for new data, so all we need to do is provide a Datum when asked. To keep things simple, we'll implement just net.solarnetwork.node.DatumDataSource, which looks like this:

public interface DatumDataSource<T extends Datum> extends Identifiable {

	/**
	 * Get the class supported by this DataSource.
	 *
	 * @return class
	 */
	Class<? extends T> getDatumType();

	/**
	 * Read the current value from the data source, returning as an unpersisted
	 * {@link Datum} object.
	 *
	 * @return Datum
	 */
	T readCurrentDatum();

}

You'll notice that DatumDataSource extends net.solarnetwork.node.Identifiable, which is another pretty simple API used by the SolarNode framework to support selecting specific service instances for specific tasks. We won't worry much about this API for the purpose of this guide, but the API looks like this:

public interface Identifiable {

	/**
	 * Get a unique identifier for this service. This should be meaningful to
	 * the service implementation.
	 *
	 * @return unique identifier (should never be <em>null</em>)
	 */
	String getUID();

	/**
	 * Get a grouping identifier for this service. This should be meaningful to
	 * the service implementation.
	 *
	 * @return a group identifier, or <em>null</em> if not part of any group
	 */
	String getGroupUID();

}

Implementation

To recap what we have learned, we know we want to:

write a class that implements net.solarnetwork.node.DatumDataSource, that performs the work of reading sample data from our Foobar inverter
this class should create instances of net.solarnetwork.node.domain.GeneralNodePVEnergyDatum from the sampled data

OSGi package imports

This guide will not go into details on OSGi, but for our bundle to have access to to the DatumDataSource interface and GeneralNodePVEnergyDatum class, our bundle must declare its desire to use the packages that interface and class are defined in. Open the META-INF/MANIFEST.MF file in Eclipse. By default this opens in Eclipse's Manifest Editor. Click on the Dependencies tab (located at the bottom of the editor).

Click on the Add... button, and use the search field to add the following packages:

net.solarnetwork.node
net.solarnetwork.node.domain
net.solarnetwork.node.support
org.slf4j

Eclipse will also default to using the version of the packages available in your workspace as the minimum required package version, which is generally fine. If you click on the MANIFEST.MF tab at the bottom of the manifest editor you can see the raw source of the MANIFEST.MF file, which should look like this:

You'll notice that all the values you filled in when you first created the project show up here.

Create `DatumDataSource` class

Now we're ready to create our DatumDataSource implementation. We'll follow SolarNode conventions and create our class within a package named after our bundle ID (and Eclipse project name): net.solarnetwork.node.example.datum_capture.FoobarDatumDataSource. Right-click on the project in Eclipse and select New > Class. Fill in the appropriate package and name, and add the net.solarnetwork.node.DatumDataSource<net.solarnetwork.node.domain.GeneralNodePVEnergyDatum> interface, like this:

Eclipse will create stubs for the methods you're required to implement. Ignore the Identifiable API methods at this point. The getDatumType() should simply return the GeneralNodePVEnergyDatum class, so after writing that line the class should look like this:

/**
 * Implementation of {@link DatumDataSource} for Foobar inverter power.
 *
 * @author matt
 * @version 1.0
 */
public class FoobarDatumDataSource implements DatumDataSource<GeneralNodePVEnergyDatum> {

	// getUID() and getGroupUID() methods ignored for now...

	@Override
	public Class<? extends GeneralNodePVEnergyDatum> getDatumType() {
		return GeneralNodePVEnergyDatum.class;
	}

	@Override
	public GeneralNodePVEnergyDatum readCurrentDatum() {
		// TODO Auto-generated method stub
		return null;
	}

}

Implement logic to "sample" data

So far we've just been implementing SolarNode scaffolding. Now it's time to implement the actual code to read samples from the Foobar inverter and return the data as a GeneralNodePVEnergyDatum instance. Our implementation is fictional, so we'll simply return fake instantaneous watt readings and fake accumulating watt-hour readings. First, to provide data that models a real-world inverter, we'll add a counter to keep track of the accumulated watt-hours generated, and a configurable property to assign a sourceId value to the returned PowerDatum instances:

	private final AtomicLong wattHourReading = new AtomicLong(0);

	private String sourceId = "Inverter1";

	public void setSourceId(String sourceId) {
		this.sourceId = sourceId;
	}

Then, let's implement the readCurrentDatum() method to return some data modeled after a 1kW PV system, returning randomized data:

	@Override
	public GeneralNodePVEnergyDatum readCurrentDatum() {
		// our inverter is a 1kW system, let's produce a random value between 0-1000
		int watts = (int) Math.round(Math.random() * 1000.0);

		// we'll increment our Wh reading by a random amount between 0-15, with
		// the assumption we will read samples once per minute
		long wattHours = wattHourReading.addAndGet(Math.round(Math.random() * 15.0));

		GeneralNodePVEnergyDatum datum = new GeneralNodePVEnergyDatum();
		datum.setCreated(new Date());
		datum.setWatts(watts);
		datum.setWattHourReading(wattHours);
		datum.setSourceId(sourceId);
		return datum;
	}

Unit testing

We've got our DatumDataSource implemented now, let's write some unit tests. A nice way to write unit tests is to create a new OSGi fragment bundle project. A fragment bundle attaches to a host bundle, inheriting all the package imports of the host, but can add additional classes and imports. This cleanly separates the unit test code from the real code, and the unit test bundles need not be deployed on your SolarNode, minimizing the footprint on the node.

Start by creating a new fragment project in Eclipse by selecting File > New > Project....

From the dialog window that appears, select Fragment Plug-in Project and click Next. In the next screen, fill in the following details:

Project name - net.solarnetwork.node.power.foobar.test (our host project name with .test appended)
Output folder - set to build/eclipse so we can support standard SolarNode Ant-based builds later on.
Target Platform - set to a standard OSGi framework

Click the Next > button, and on the next screen fill in the OSGi bundle information. The most important field is the Host Plug-in ID which should be set to net.solarnetwork.node.power.foobar:

Finally click the Finish button and Eclipse will create the project and open the project's manifest editor. We'll be unit testing with JUnit 4 and making use of the net.solarnetwork.node.test project, and as such need to add the following package imports:

net.solarnetwork.node.test
org.junit
org.junit.runner

Save the changes, then right-click on the project in Eclipse and select New > Class to create our unit test for the FoobarDatumDataSource class. Fill in the appropriate package and name the class FoobarDatumDataSourceTests:

For the implementation, we'll add a setup method to configure an instance of the FoobarDatumDataSource class, and a method that verifies the data source produces data as expected. We'll use standard JUnit 4 annotations @Before and @Test:

public class FoobarDatumDataSourceTests {

	private static final String TEST_SOURCE_ID = "Test";

	private FoobarDatumDataSource service;

	private final Logger log = LoggerFactory.getLogger(getClass());

	@Before
	public void setup() {
		service = new FoobarDatumDataSource();
		service.setSourceId(TEST_SOURCE_ID);
	}

	@Test
	public void readOneDatum() {
		GeneralNodePVEnergyDatum d = service.readCurrentDatum();
		log.debug("Got datum: {}", d);

		// datum should not be null
		Assert.assertNotNull("Current datum", d);

		// the source ID should be what we configured in setup()
		Assert.assertEquals("Source ID", TEST_SOURCE_ID, d.getSourceId());

		// the watts and watt hours values should not be null, but we don't know
		// exactly what they'll be because they produce random data
		Assert.assertNotNull("Watts", d.getWatts());
		Assert.assertNotNull("Watt hours", d.getWattHourReading());
		Assert.assertTrue("Watt range", d.getWatts() >= 0 && d.getWatts() <= 1000);
		Assert.assertTrue("Watt hour range",
				d.getWattHourReading() >= 0L && d.getWattHourReading() <= 15L);
	}

}

Now you can right-click on the class in Eclipse and select Run As > JUnit Test. The test should run and complete without errors.

Now let's add another unit test method that verifies that the watt hour reading does not decrease each time we call readCurrentDatum(), to give us confidence the class is modeling a real inverter sensibly. We'll also add a debug log statement, to see how logging can be used. Add this following method:

@Test
public void readSeveralDatum() {
	long lastWattHourReading = 0;
	for ( int i = 0; i < 10; i++ ) {
		PowerDatum d = service.readCurrentDatum();
		Assert.assertNotNull("Current datum", d);
		Assert.assertEquals("Source ID", TEST_SOURCE_ID, d.getSourceId());
		Assert.assertNotNull("Watts", d.getWatts());
		Assert.assertNotNull("Watt hours", d.getWattHourReading());
		Assert.assertTrue("Watt range", d.getWatts() >= 0 && d.getWatts() <= 1000);
		log.debug("Got Wh reading: {}", d.getWattHourReading());
		Assert.assertTrue("Watt hour range",
				d.getWattHourReading() >= lastWattHourReading
						&& d.getWattHourReading() <= lastWattHourReading + 15L);
		lastWattHourReading = d.getWattHourReading();
	}
}

Re-run the unit test class in Eclipse, and both tests should pass. Click on the Console tab in Eclipse to see the generated logging output. You might see the following:

log4j:WARN No appenders could be found for logger (net.solarnetwork.node.power.foobar.test.FoobarDatumDataSourceTests).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

That means you haven't configured the necessary log4j.properties file for logging to work properly. In the net.solarnetwork.node.test project, copy the environment/example/log4j.properties file into the environment/local directory. This will enable TRACE level logging for SolarNode classes by default. Re-run the unit tests, and you should see output similar to this:

Aug-16 11:53:58 TRACE - Got datum: GeneralNodePVEnergyDatum{sourceId=Test,samples={watts=898, wattHours=45}}
Aug-16 11:53:58 DEBUG - Got Wh reading: 45
Aug-16 11:53:58 TRACE - Got datum: GeneralNodePVEnergyDatum{sourceId=Test,samples={watts=780, wattHours=54}}
Aug-16 11:53:58 DEBUG - Got Wh reading: 54
Aug-16 11:53:58 TRACE - Got datum: GeneralNodePVEnergyDatum{sourceId=Test,samples={watts=151, wattHours=68}}
Aug-16 11:53:58 DEBUG - Got Wh reading: 68

Continue...

We've got some working code now, but it still isn't integrated into SolarNode. Continue to part two to see how that works.

SolarNode Datum Capture Plug-in
- Part 2
- Part 3
- Part 4

Datum Capture Part 1

SolarNode Datum Capture Plug-in Example

Eclipse Setup

Create new Plug-in project for Foobar Power

SolarNode datum

SolarNode datum data source

Implementation

OSGi package imports

Create DatumDataSource class

Implement logic to "sample" data

Unit testing

Continue...

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Create `DatumDataSource` class