Scenario.

I wanted to create a bar chart which has a dynamic width (expand or contract) depending on the amount of categories a user selected from a multi value list parameter. This would ensure that the chart didn’t become squashed and unreadable. in this tutorial I will be using a data set which represents population by countries.

Data Set.

Below is an example of the data set I will be using for this tutorial, as you can see each country has a population (you can download a spreadsheet version of the sample data at the end of this tutorial).

* Population is calculated in the millions

Step 1 – Data Source.

Start PRD and create a new report. Right click on the Data Sets icon and select the JDBC option from the menu.

After selecting the JDBC option a JDBC Data Source window will appear. I have already created a connection to my database which includes the data that I will be working on, in this example it is called Geography.

Once you have a working connection to your database you are now ready to create your queries.

Step 2 – Queries.

This report contains two separate queries:

1. qCountries : Retrieves a distinct list of countries, this will be used for the countries parameter
2. qDataset : Retrieves the population for the countries selected by a user

pCountries.

The qCountries query needs to return a distinct list of countries, the SQL query looks like this:

SELECT   DISTINCT COUNTRY
FROM     dataset

A screen shot of the JDBC Data Source window of the qCountries query looks like this:

qDataset.

The qDataset query needs to return the population for countries which the user has selected, the SQL query looks like this (this query will need to use a parameter which I will add in later):

-- Use a sub query to calculate the total amount of countries selected
SELECT    t1.COUNTRY AS COUNTRY,
               SUM(t1.POPULATION) AS POPULATION,
               SUM(t2.COUNTRY_COUNT) AS COUNTRY_COUNT
FROM      dataset t1,
          (
              SELECT COUNT(*) AS COUNTRY_COUNT
              FROM dataset
              GROUP BY COUNTRY
          ) t2
GROUP BY COUNTRY

A screen shot of the JDBC Data Source window with the qDataset query looks like this:

Step 3 – Parameter.

There is only one (multi value list) parameter in this report which will display a distinct list of countries and be populated by the qCountries query. To create a new parameter right click on the Parameters option under the Data tab and click the Add Parameter… option:

This will bring up the Add Parameter window. After filling out all the options the Add Parameter window now looks like this:

Where the options and values are:

• Name : This is the name of the parameter, this is used within report queries i.e. the WHERE clause of qDataset
• Label : The text which sits next to the parameter on the report i.e. Select one or more countries
• Value Type : The data type of the sCountries value i.e. COUNTRY is a String
• Mandatory : For the report to run a parameter value must be selected i.e. Checked (true)
• Display Type : The parameter type i.e. Multi Value List
• Value : The parameters value which will used within queries etc. i.e. the field COUNTRY
• Display Name : What the user will see the parameter value as i.e. the field COUNTRY
• Visible items : The amount of items which are shown to the user in the multi value list parameter i.e. 3

Click OK to save the parameter, it should now be visible under the Parameters item under the Data tab.

After adding the sCountries parameter the next step is to use the parameter as part of the qDataset WHERE clause. Right click on the qDataset item under the Data tab and click the Edit Query option:

Modify the qDataset query so that it now includes the sCountries parameter:

-- Use a sub query to calculate the total amount of countries selected
SELECT    t1.COUNTRY AS COUNTRY,
               SUM(t1.POPULATION) AS POPULATION,
               SUM(t2.COUNTRY_COUNT) AS COUNTRY_COUNT
FROM      dataset t1,
          (
              SELECT COUNT(*) AS COUNTRY_COUNT
              FROM dataset
              WHERE COUNTRY IN (${sCountries})
              GROUP BY COUNTRY
          ) t2
WHERE COUNTRY IN (${sCountries})
GROUP BY COUNTRY

Step 4 – Bar Chart.

The next step is to create a bar chart whose width is dynamic based on the amount of countries a user selects. To get started drag and drop the chart object onto the report header.

As you can see the width is fixed at the moment but before I start to make any changes to the chart’s width I will need to configure my chart to display our data and see how it becomes unreadable when we increase the amount of categories on the x axis. Double click on the chart and to bring up the Edit Chart window.

By default the chart is set to a bar chart and without going through every available option I made the following changes to represent my data set:

• chart-title : Chart title i.e. Population of Countries (optional)
• gridlines : Are the vertical and horizontal grid markers i.e. Set to False (optional)
• x-axis-title : Label of the x-axis i.e Countries
• x-axis-label-rotation : Rotation of the x-axis label i.e. 90
• y-axis-title : Label of the y-axis i.e. Population (millions)
• category-column : Field which will be on the x-axis i.e. COUNTRY
• value-columns : Field which populates the y-axis i.e. POPULATION
• series-by-field : Field which defines the series i.e. COUNTRY

After previewing the chart with all the countries selected you can see that the output is squashed and unreadable.

One way around this is to make the width (or height) of your chart dynamic by the amount of countries the user selects from the multi value list parameter. Select the chart and on the right hand side under the Style tab locate the size & position option and click the icon under the Formula column. This will display an Expression window, click on the icon which will then open the Formula Editor window.

The formula below retrieves the COUNTRY_COUNT field and multiplies it by the width of the chart which suits one category (country) on the x axis, which in this example is 150.

# If there is only one country selected then use default width otherwise calculate width
=IF([COUNTRY_COUNT]=1;150;[COUNTRY_COUNT]*150)

Save the formula. Now you are ready to preview the report.

Step 5 – Preview.

Click on the preview icon and select one country from the multi value list parameter, as you can see the width is set to our default width, 150.

Selecting two countries from the multi value list parameter will expand the width to 300.

Selecting four countries from the multi list parameter will expand the width to 600.

So that is how you create a chart which can expand or contract depending on the amount of categories displayed on the x axis.

Downloads.

Final Notes.

A couple of things to note about the method I used:

• The query to generate the amount of selected countries uses a sub query and could cause performance issues with large data sets etc.
• This method can be used for both the width and height and on any other object within PRD i.e. shapes, labels, pictures etc.

Hopefully this comes in handy when you start handling large data sets and charts!

Enjoy!

I have 40 sale campaigns. I need to put together a quick presentation (as a PDF) which will show 2 key metrics (New and Loss revenue) for each of the 40 sale campaigns. Do you know of a quick way I can produce this presentation?

I thought this would the perfect opportunity to show off Pentaho Report Designer (PRD) as an alternative to Mircosoft Excel, Access or other big BI vendors. Below is a step by step guide on how I accomplished this.

Scenario.

The above statement sums up the users requirements, visually the the report needed to look like this:

After I knew the structure of the report I got the user to export the data into an Excel Worksheet – this was going to be my data source – after this it took me only 5 minutes to create the presentation using PRD.

Data Source.

The end user was able to produce a raw extract of the data which the report needed to be based on. Here is a preview of the data:

• Campaign: Campaign identifier which duplicates for each week row
• Week: Week ending date which is for 2010
• New_09, Loss_09: Revenue for 2009
• New_10, Loss_10: Revenue for 2010

To use an Excel worksheet as a data source make sure the Data tab is active, right click on the Data Sets icon and select the Table option:

After clicking the Table option a Table Datasource Editor window is displayed:

Step 1. Create a Query.

The first step is to create a query which will store the data set. Click on the  (Add Query) icon and then specify a name, in this example I’m naming the query dataset.

Step 2. Select the Excel Worksheet.

The next step is to select the Excel worksheet which contains the data set. Click on the  icon located in the bottom left hand corner to open up a file browser window, select the Excel file which contains the data set and click OK. Your data set should now be visible (as my Excel worksheet has headers I have checked the Use First Row as Header option).

At the moment you can only use .XLS files so if you are using Office 2007+ make sure you save the extension as .XLS instead of .XLSX and if your .XLS file contains formulas they will not be recognised by PRD so be sure to paste special as values.

Step 3. Manipulate the Data.

You can add empty rows and columns or remove rows and columns by using these icons:

You are also able to change the column headers and data type by double clicking on the column header:

In this example I didn’t need to make any modifications to the data however I did have trouble converting the Week_Ending field into a date format so in the Excel worksheet I converted it into a string using the TEXT() function – if anyone knows how to get this working please lleave a comment.

Once you are happy with the data set click the OK button. The data set should now be visible as fields under the Data Sets > Table > dataset icon under the Data tab:

Building the Report.

The next step was to build the report, from the report structure (above) I knew I had to create one group Campaign, and within that group add 2 charts which represented the New and Loss Revenue for 2009 and 2010.

Step 1. Add the Group.

To get started add the Campaign field as a group by right clicking on the Groups item under the Structure tab and selecting the Edit Group .. option:

After clicking the Edit Group .. option an Add Group window will appear, specify a name for the group i.e. Campaign and using the arrows move the field over which defines the group which in this example is Campaign and click OK when done.

The Campaign field will now be defined under the Attributes tab for the Group item:

The next step is to make the Group Header and Details Header are visible on the report canvass. Click on the Groups > Group Header item under the Structure tab and and under the Attributes tab uncheck the hide-on-canvass option:

Repeat the above steps for the Groups > Details Body > Details Header item as well:

Step 2. Functions.

I will need to create three functions; one which will convert the Week_Ending field into a date and the other two will be display dynamic chart titles.

Converting the Date.

As I mentioned above I had some issues with letting PRD convert the Week_Ending field into a date from the Table Datasource Editor, so to get around this I created a formula which would take the Week_Ending field (as a string) and convert it into a Date.

To do this create a new function by right clicking on the Functions option under the Data tab and click the Add Function… option.

From the Add Function… window expand the Common folder and select the Open Formula option and click the Add button:

This will add an Open Formula function under the Functions option (seen under the Data tab) :

To change the name of the function (named by default as FormulaExpression1) click on the function and in the area below change the value that is set for the Function Name, in this example I named the function New_Week_Ending:

To edit the formula of the function click on the  icon which is displayed when you click in the Formula value area, this will display a Formula Editor window:

As my date is in the current format YYYY-MM-DD I needed to use the DATE function with a combination of LEFT, MID and RIGHT functions to convert it into a date. I won’t go into too much detail with this section as all these functions are pretty self explanatory and are well documented in the Formula Editor but the final formula looked like this:

=DATE(LEFT([Week_Ending];4);MID([Week_Ending];6;2);RIGHT([Week_Ending];2))

Click the OK button to save the New_Week_Ending formula. I have just created a function which will convert the Week_Ending (string) field into a New_Week_Ending (date) field so that the X-Y Chart TimeSeries Collector will now recognise it as a date.

Dynamic Chart Titles.

Each chart needs to have a dynamic title which will output the campaign and the measure which is represented i.e. 1 – New Revenue ($). As I have two charts for each campaign I will need to create two functions: one for the New Revenue chart and one for the Loss Revenue chart.

Following the steps outlined above to create two new functions (with the names sNewChart and sLossChart) using the Open Formula type and specify the formulas to be:

# New Revenue Charts (sNewChart)
=[Campaign] & " - New Revenue ($)"
# Loss Revenue Charts (sLossChart)
=[Campaign] & " - Loss Revenue ($)"

I have now created three functions:

Step 2. Add the Group Header.

To add the Campaign as the Group Header drag and drop the Campaign field from the Data tab onto the Group Header area and apply some formatting:

Step 3. Add the Charts.

Each chart needed to look similar to this mockup (I have already created the Chart Title and New_Week_Ending functions):

Drag and drop a chart object from the panel located on the left hand side onto the Details Header section and double click the object to open up the Edit Chart (properties) window . To create a X-Y Chart which uses a TimeSeries Collector make sure you Edit Chart (properties) window looks like this:

After setting up all the options the Edit Chart (properties) window now look like this (for the New Revenue measure):

• Title
  • chart-title-field: Field or function which is the chart title i.e. sNewChart or sLossChart
• Primary DataSource
  • Common
    
    • category-time-option: Value of the x-axis, this must match the amount of series you have in the value-column option i.e. New_Week_Ending (twice)
    • time-period-type: Type of period the above field is i.e. Week and New_Week_Ending is an increment of weeks
    • value-column: Field(s) which contain the y-axis, you can specify more than one as there can be multiple series i.e. New_09 and New_10 or Loss_09 and Loss_10
  • Series
    • series-by-value: Name of the each series specified for the above option i.e. 2009 and 2010
  • Group
    • reset-group: Reset the charts after a specific group(s) i.e. Campaign

There was alot of extra formatting on the charts so be sure to check out the available .PRPT file if you interested in what they look like.

After I created the chart for New Revenue I just copied and pasted the chart object and made changes to the properties to create the Loss Revenue chart:

The final report structure looks like this:

There are a lot of options in the charting library of PRD so I suggest you pick up Will Gorman’s book Pentaho Reporting 3.5 for Java Developers which has a couple of chapters which cover the charting library in detail.

Step 4. Add the Page Break.

The last step was to add a page break after each Campaign group. To do this click on the Details Body item under the Structure tab and check the pagebreak-after option:

Output.

After previewing the report the final output as a PDF looks like this (you can download the actual PDF file below):

Downloads.

Final Notes.

This post is very detailed as it covers some interesting areas but as this was an ad hoc request it took me under 5 minutes to whip together this report and have it in the user’s inbox. Do you have any examples of how you handled ad hoc requests with PRD? If so, leave a comment or send me an email.

One user converted n to go! Enjoy.

Below is a list of open source and free tools which support or are dedicated to creating Sparklines or Microcharts.

Jquery Sparkline

This jQuery plugin generates Sparklines (small inline charts) directly in the browser using data supplied either inline in the HTML, or via javascript.

Last updated : 14th of September, 2009
Requirements : jQuery 1.2+
Supported chart types : Line, Bar, Tristate, Discrete, Bullet, Pie, Box

Sparklines for Excel

A set of free User Defined Functions for Excel to create Sparklines. This is probably the most flexible and up to date sparkline implementation for Excel 2003 and 2007.

Last updated : 02/04/2010 (Excel 2003) and 01/25/2010 (Excel 2007)
Requirements : Excel 2003 and 2007
Supported chart types : Line, Area, Bar, Horizon, Vari, Bullet, Scale, Pie, Stacked, Box Plot, Spread, Stripe

Pentaho Report Designer

The ability to create Sparklines is now a feature of Pentaho Report Designer 3.5+.

Last updated : 27/10/2009
Requirements : Pentaho Report Designer 3.5+
Supported chart types : Line, Bar

Google Visulization API

Single or multiple sparklines that are rendered with images using the Google Charts API. The images are contained in a HTML table.

The Google Chart API is an extremely simple tool that lets you easily create a chart from some data and embed it in a webpage. You embed the data and formatting parameters in an HTTP request, and Google returns a PNG image of the chart. Many types of chart are supported, and by making the request into an image tag you can simply include the chart in a webpage.

Last updated : Unknown
Requirements : Flash
Supported chart types : Line

Sparkline PHP Graphing Library

A PHP based graphing library to create Sparklines.

Last updated : 10th of November, 2007
Requirements : PHP 4.0.6+, GD 2.0 built as a PHP module
Supported chart types : Line, Bar

Sparklines for Java

Sparklines for java is a package that contains a Servlet that can be added to a web project to create sparklines based on request parameters. It also includes a taglib for easily formatting the required parameters. There’s also an API to directly create a BufferedImage in any Java code.

Last updated : 3/11/2007
Requirements : Java
Supported chart types : Line, Bar

Sparklines (Ruby)

A library for generating small sparkline graphs from Ruby. Use it in desktop apps or with Ruby on Rails.

Last updated : 17/02/2009
Requirements : Ruby
Supported chart types : Line, Bar, Bullet, Pie, Discrete, Area

NanoCharts

NanoCharts is an Excel 2003 supported add-in where users can add in Sparklines into cells.

Last updated : 08/10/2008
Requirements : Excel 2003
Supported chart types : Line, Bar, Dot

Sparklines Excel Add-in – TinyGraphs

TinyGraphs is an open source add-in for Excel that creates sparklines, mini column graphs, and area graphs from a row of data. It generates beautiful tiny graphics that are as small as a cell and is useful for visualizing large quantities of data, such as stock prices and exchange rates.

Last updated : Unknown
Requirements : Excel 2000, 2003 and 2007
Supported chart types : Line, Bar, Area

Tufte Sparklines Photoshop Script

The Tufte Sparklines Photoshop script automates the production of sparklines. It works by reading a data series from a text file and plotting a sparkline image as a Photoshop path, and then stroking it with the pencil tool. This produces a bitmap image which can then be cut-and-pasted into the target document.

Last updated : 24/02/2008
Requirements : Adobe Photoshop CS, CS2, CS3
Supported chart types : Line

sparklines.js

parklines.js is a simple javascript utility for building sparklines. It has support for displaying lines at percentage points and at fixed values in terms of your data set.

Last updated : Unknown
Requirements : Processing.js 0.4
Supported chart types : Line

Know of anymore? Let me know and I’ll add them to this list.

Enjoy.

VisualPlace is a six month Proof of Concept testing innovative ways of capturing, exploring and visualising government information and services through maps. VisualPlace brings together information from a wide variety of sources from service locations to demographic data to allow you to create custom maps.

VisualPlace uses a Silverlight powered application known as VisualFusion (developed by IDV Solutions) and can map data from 2006 National Census, Government Service locations and even Tram and Train station and lines. For example here is a mashup of “People who travelled to work by Public Transport as a proportion of All Employed People” (shaded areas where yellow are high percentages and blue are low percentage) against “Train Stations” (green markers):

The only thing that annoys me is that if the Governement Services related data is available why not make it easily accessible to the public via an API or data.government.gov.au. If you would like to gain access to Demographic (Census) data visit the Australian Bureau of Statistics website.

Be sure to check it out quick as it will be taken down by July 2010.

Enjoy.