jCleanCim release notes

Readme file


Release notes for jCleanCim-02v01, 2016-07-23


New home

Since 2016, jCleanCim has new home at my web space - which is accessible also to non-CIMug members.

New features

Hyperlinks in the Word document

Credit: Prototype by Laurent Guise (thanks!) has inspired the implementation in this release.

This (by IEC 61850 community) long awaited feature is in and it is controlled by a new configuration option, docgen.word.useHyperlinks . To not incure performance penalty, the default is 'disabled' (i.e., you need to set it to true explicitly .

The hyperlinks are introduced for those classes and enumeration literals that have been printed in the same document. For example:

New MS Word document placeholders

In this release, we have added three new placeholders for MS Word document generation:

We have also added examples in the base-small-template.docx to show how to use the new placeholders.

Support for EA Enumeration type (that discards class with "enumeration" stereotype)

In more recent versions of EA, on XMI import, EA refuses to import the original class with stereotype "enumeration", but transforms it automatically to the new Enumeration type. We found no way to force EA to keep enumeration stereotype imported through XMI. The result was that the enumeration classes were then not recognised in jCleanCim as enumerated types and one had to manually add enumeration stereotype.

This release provides proper handling for this case, as well - i.e., the EA Enumeration type is now recognised as enumerated type, the same way as a class with "enumeration" stereotype.

Performance changes in release 02v01

None. Note however that newer versions of Enterprise Architect seem to always be somewhat slower than the older ones (we noticed the slow down in diagram generation).

Potential backwards compatibility breaking changes for the application user

Potential forwards incompatibility for the application user: old jCleanCim with new EA

Other functional changes

Validation rules:

Bug fixes:

Known issues / limitations:

Implementation and packaging:

Documentation:


Release notes for jCleanCim-02v00, 2015-05-19


Licensing changes

From this release, the license terms have changed, see disclaimer. There should be no big impact if you were using it the same way as I did :-)

Distribution changes

Producing the distribution "Eclipse project archive jCleanCim-[version]-eclipse.zip" has been discontinued, because it's a manual and time consuming process. The same functionality is available by using the "Source distribution jCleanCim-[version]-src.zip" - by importing simply the eclipse project file from un unzipped directory.

New features

This is a release with incremental changes and no "real" new feature.

Performance improvements in release 02v00

None.

Potential backwards compatibility breaking changes for the application user

Potential forwards incompatibility for the application user: old jCleanCim with new EA

Other functional changes

Validation rules:

Bug fixes:

Known issues / limitations:

Implementation and packaging:

Documentation:


Release notes for jCleanCim-01v10, 2014-05-20


New features

This release does not introduce big new features, just a couple of formatting changes (for document generation), and a couple of new validation rules. For details, see the sections below.

Performance improvements in release 01v10

None.

Potential backwards compatibility breaking changes for the application user

Other functional changes

Validation rules:

Bug fixes:

Known issues / limitations:

Implementation and packaging:

Documentation:


Release notes for jCleanCim-01v09, 2013-05-11


The main focus for this release was on validation, and some formatting specials for IEC61850 document generation.

New features

Validation report available as .csv

If you are keen on keeping your model clean (the initial very purpose of this application), then you'll appreciate the new validation report, now generated every time you run with validation enabled. The report gets stored in the project's output directory as a comma-separated file problemsReport-[model-name].csv . You can then open it with a spreadsheet application, format as you like and do filtering of model issues found.

Disabling logging of ellapsed time

From this release it is possible to control the time logging aspect of the run by means of the new configuration option app.skipTiming . When set to true , the ellapsed time currently logged upon completion of the major steps will not be logged anymore. So, instead of e.g.:

  [main] INFO  time=[0:00:10.250] completed all configured steps - exiting
  

the log will look so:

  [main] INFO  completed all configured steps - exiting
  

This may be useful to allow for seemless text comparison of two consecutive runs of the application, such as when you e.g. apply changes to the model and re-run validation or statistics, then want to compare the two log runs.

By default, the timing of major steps gets logged as usual.

Performance improvements in release 01v09

There are two places with performance improvements in this release, but they are probably not very noticeable, because the task duration initially was of the order of seconds or below:

Potential backwards compatibility breaking changes for the application user

None. The only difference in configuration file that you may notice is that only a small number of rules has been kept as ignored (to decrease the noise until important problems get fixed). In every case, you should try to fix the existing reported problems, then remove the ignored rules from the configuration file one by one (and fix those problems).

Note that from this release, you will need Java 7 or higher.

Other functional changes

Validation rules:

Bug fixes:

Known issues / limitations:

Implementation and packaging:

Documentation:


jCleanCim-01v08a, 2013-02-10


Note: Version 01v08 should have been called -beta and it has not been released, but was used to generate auto-docs of various 61850 documents in last 2 weeks (plans were to release that one, but with lack of time, the deadline has been missed...). Below, I use 01v08 to refer to this 01v08a as well, and 01v08a only for the fixes applied the day of this official release.

New features

Selective XMI format export

We have added the means to configure which XMI formats to export, by setting the new configuration option xmiexport.dialects to one or more comma-separated values. If nothing is specified, we keep the (almost) default behaviour, to export XMI files in all supported formats, except for cimtool ; see config.properties for recognised values.

Note: The reason to not export cimtool by default along with others is that CIMTool latest releases (1.9.6+) simply don't require .xmi file anymore, and can work directly with the .eap model file. Also, the CIM model exported as .xmi has more than 80MB, so we can be more network and disk friendly. If you however need to export it, just specify it in the new configuration option xmiexport.dialects.

Fast loading of .eap file (if no need to export diagrams or XMI)

We have added as an option, an extremely fast implementation for reading the .eap model file (thanks to Arnold deVos for the hint on a pure java library for reading M$ Access files: Jackcess). Our big standard UML models can now load in a couple of seconds, and this functionality is completely independent of the Enterprise Architect libraries. Consequently, this is totally platform independent and can run on any OS.

The limitation of this new implementation is that we cannot export diagrams or XMI, because those exports are performed by the EA repository API (not used with this implementation). Still, most of the time, you'll be doing model edit-validate-fix cycles, and this new implementation supports perfectly that use case: you can validate very quickly what you just edited.

In order to support the switch among now three possible implementations for the .eap model loading, we decided to replace the boolean configuration option model.useSql from 01v07 with the new option: model.builder , which can take one of three values: db, sqlxml or japi . Table below gives a couple of characteristics to facilitate the choice of appropriate builder for your actual task.

model.builder= db sqlxml japi
how it reads .eap model file as Access DB as .eap repository, queries (SQL) + resultset (XML) EA Java API
speed as fast as it gets opening .eap very slow, iterating model very fast opening .eap very slow, iterating model very slow
needs ea.jar + ea.dll no yes yes
bound to M$ Windows no yes yes
can export UML diagrams never yes (if docgen.on=true) yes (if docgen.on=true)
can export XMI never yes (if xmiexport.on=true) yes (if xmiexport.on=true)

Performance improvements in release 01v08

B-I-G !

The following numbers have been traced from runs on Lenovo ThinkPad T410 (i5 CPU, 3GB memory), 64-bit Windows 7 Enterprise Edition, Office 2010, Enterprise Architect 9, 32-bit Java 1.7.0_13.

The first improvement is related to reading data from .eap UML repository:

Note that for diagram export ( docgen.on=true ), it takes the same time with both model.builder=sqlxml and model.builder=japi , i.e., some 100-300ms per normative diagram exported.

The UML models used, referred to in the table below:

EA-dependent operation 01v07

useSql=false / useSql=true / -

01v08

japi / sqlxml / db

Open .eap file of any size (SSD hard disk) 5-10 sec / 5-10 sec / - 5-10 sec / 5-10 sec / 0.14-0.25 sec
Read CIM.eap
(docgen.on=true: ~355 exported diagrams)
3 min / 29 sec / -
(+50 sec)
3 min / 20 sec / 7.5 sec
(+50 sec)
Read IEC61850.eap
(docgen.on=true: ~200 exported diagrams)
2.8 min / 26 sec / -
(+32 sec)
2.8 min / 17 sec / 6.4 sec
(+32 sec)
Read base-small.eap
(docgen.on=true: ~93 exported diagrams)
34 sec / 10 sec / -
(+12 sec)
34 sec / 8 sec / 1.3 sec
(+12 sec)

Performance for MS Word document generation has been significantly improved for very large documents with non-formatted tables (thanks to Andre Maizener's tip on ctrl-F11 !). This will work only on the template that has been saved as native Office 2007/2010 document (.docx), without compatibility enabled, because in this case only, it is possible to programmatically disable field updates.

However, for documents with fancy table formatting (and very large tables), there is very little performance improvement (beauty costs :-).

Note that the "magic number" 12 is not anymore magic. We have found best numbers for our running environment, but you may want to try other numbers that may be better for your local running environment (until we complete the new implementation, based on working with simple XML instead of COM API).

MS Word doc generation 01v07
(docgen.saveReopenEvery) duration
01v08
(docgen.saveReopenEvery) duration
speed improvement
IEC61970-301 Base
(637 tables, 67 figures)
(12) 43.3 min (27) 9.5 min 4.6 x
IEC61968-11
(378 tables, 37 figures)
(16) 12.1 min (24) 4 min 3.2 x
IEC62325-301
(637 tables, 40 figures)
(16) 36.5 min (24) 7.9 min 4.6 x
IEC61970-302 Dynamics
(387 tables, 178 figures)
(12) 18.5 min (27) 6 min 3.1 x
IEC61850-7-3 including a subset of IEC61850-7-2, with special table formatting
(134 tables, 31 figures)
(12) 4.4* min (5) 3.9 min 1.1 x*
IEC61850-7-4, with special table formatting
(244 huge tables, 40 figures)
(12) 41.4 min (27) 27.5 min 1.5 x
base-small
(102 tables, 27 figures)
(12) 0.8 min (5) 0.7 min 1.1 x**

* jCleanCim-01v07 does not support printing tracking CDC's attributes (special FC), so there are some 15 tables that are much smaller when printing with 01v07 than with 01v08.

** This document is relatively small, so the cumulative effect of appending tables does now have time to show up.

Potential backwards compatibility breaking changes for the application user

Other functional changes

New validation rules:

Bug fixes:

Known issues / limitations:

Implementation and packaging:

Documentation:


jCleanCim-01v07, 2012-08-26


New features

Fast EA model reading

In this release, we tried again to improve performance, because our standard UML models grow and the COM APIs (EA and MS Word) are terribly slow.

On EA side, we introduced a new configuration option model.useSql that is false/empty by default, so it does not change the previous behaviour. When set to true, we use SQL bulk queries and process the returned XML dataset with XPath, instead of calling the EA API methods with their extremely slow collections and chatty SQL queries. This improves the performance of reading the model from EA repository for an order of magnitude - see numbers in the performance section below. The speed up is a bit less significant when document generation is enabled, because this new implementation does not concern diagram export (which is inherently slow: 300 - 500 ms per diagram).

Exporting XMI files

This is a new top level feature (sibling to validation, statistics and doc generation), and is introduced with the new configuration property xmiexport.on that is false/empty by default, so it does not change the previous behaviour (and because this takes time: EA is very slow at exporting to XMI in every set-up).

As a CIM model manager, you can set xmiexport.on=true once you're ready to publish the release (i.e., you've done editing of the model and other files, fixed validation warnings and errors, etc.). Instead of the error-prone manual exporting 3 XMI files (in different XMI formats) from within an open instance of Enterprise Architect, this option will result in an automatic export of those XMI files. The principle for the output files is similar to what we do for document generation: if the XMI files already exist in the ./output directory, they are backed up and the .xmi is saved in a new file. When done, you just need to copy these 3 .xmi files into your release directory.

For IEC61850 UML model managers, at present there is no need to do any XMI export, because we don't have profiling like in CIM, and those who want to see some UML seem to be happy with Enterprise Architect. Still, this option will generate the same 3 XMI exports as for CIM models (i.e., there is no option to select only one format, or one name space, or similar).

Performance improvements in release 01v07

Release 01v07 does contain a full new implementation of alternative way of initialising in-memory model from Enterprise Architect: instead of using simply the API, we now do the bulk SQL queries, and initialise the model from the returned XML data by using XPath. Performance improvement is huge, at a cost of potential need to trick the EA model: see EA ordering errors.

The following numbers have been traced from runs on Lenovo ThinkPad T410 (i5 CPU, 3GB memory), 64-bit Windows 7 Enterprise Edition, Office 2010, Enterprise Architect 9, 32-bit Java 1.7.0_05. Performance has been significantly improved for reading from EA file; there are no changes in timing for MS Word document generation (yet):

EA-dependent operation 01v07 API vs. SQL+XML

duration - no diagram export
01v07 API vs. SQL+XML

duration - normative+doc diagram export (docgen.on)
Open .eap file of any size ~5 sec (SSD hard disk) ~5 sec (SSD hard disk)
Read full CIM model from .eap file

(~1500 classes, ~5900 attributes, ~1080 associations, ~80 dependencies, ~490 diagrams, ~250 diagrams exported when docgen.on=true)

2.9 min vs. 26 sec 3.5 min vs. 1 min

(both have +35 sec, for diagrams export)

Read full IEC61850 model from .eap file

(~1420 classes, ~5150 attributes, ~90 associations, ~300 operations, ~370 dependencies, ~200 diagrams)

2.1 min vs. 15 sec 3.4 min vs. 45 sec

MS Word doc generation 01v06

docgen.saveReopenEvery / duration
01v07

docgen.saveReopenEvery / duration
IEC61970-301, without Dynamics

(597 tables, 84 figures)
12 / 38.5 min 12 / 38.5 min
IEC61968-11

(267 tables, 62 figures)
16 / 5.5 min 16 / 5.5 min
IEC62325-301

(623 tables, 55 figures)
16 / ? min 16 / ? min
IEC61850-7-3, with special table formatting

(84 tables, 30 figures)
12 / 2.2 min (without semantic table)

12 / 2.1 min

8 / 2 min
IEC61850-7-4, with special table formatting

(234 tables, 32 figures)
12 / 41.2 min 12 / 41.2 min

16 / 38 min

20 / 35.5 min

Potential backwards compatibility breaking changes for the application user

Other functional changes

New validation rules:

Bug fixes:

Known issues / limitations:

Implementation and packaging:

Documentation:


jCleanCim-01v06, 2012-03-25


New features

This release has a fully functional new feature: XML documentation generation for both IEC 61850 and CIM name spaces. This feature was driven by IEC 61850 needs, and we provided an implementation for CIM as well (this part was trivial :-). The XML schema is included in the distribution, in the ./input directory. One schema (IECDomain.xsd) has been defined, to generate 2 XML instance files:

This split makes it possible to have a relatively lean specification exported to one XML file, with the separate pure documentation content available in one or more XML files, one per language. We obviously produce the default documentation in English, from the UML model, but the translators can produce any number of translated replicas that would all go in-line with a single specification XML file.

The schema provides support for both IEC 61850 name spaces (the way they are defined in IEC 61850-7-1) and their CIM counterparts (simply deduced from existing CIM Version classes), and by default the XML files are generated for anything found in the UML model, both IEC 61850 and CIM (as demonstrated with the mixed sample buggy model in ./input/base-small.eap). However, you can filter per IEC working group which parts you actually want auto-generated from the model. This is supported by the new configuration option docgen.xml.scope , which works exactly the same way as the other scope options - this scope applies to XML doc generation only.

Similar to MS Word document generation, XML document generation requires configuration for output files (schema name does not need be configured, it is found automatically), with the two new options: docgen.xml.outSpec holds the name of the instance file for the normative content (specification), and docgen.xml.outDoc is the name of the instance file with translatable content.

Important: In this initial release, MS Word and XML doc generation are mutually exclusive. In other words, you have to provide either the MS Word file names or XML file names through configuration options. If required, in some future release, we may support generation of it all. However, the time required for MS Word doc generation is huge (minutes or tenths of minutes) as compared to XML doc generation (seconds).

Known issue: Generated path for diagrams is wrong; this will be fixed in the next release.

Potential backwards compatibility breaking changes for the application user

In this release, these are mainly related to configuration options. We had to do heavy refactoring of existing code to support XML document generation.

Performance deterioration in release 01v06 with Office 2010 and EA 9

Release 01v06 does not contain any code modifications relevant to interaction with MS Word or Enterprise Architect. However, more recent versions of these applications seem to deteriorate performance (unless it is influenced by some PC configuration change, applied automatically by the corporate IT, which I cannot influence).

The following numbers have been traced from runs on Lenovo ThinkPad T410 (i5 CPU, 3GB memory), 32-bit Windows 7 Enterprise Edition, Office 2010, Enterprise Architect 9, Java 1.6.0_26; numbers [+##] are increase of counts or durations in the current model version as compared to the previous:

EA-dependent operation Duration - no diagram export

01v06 API
Duration - normative diagram export (docgen.on)

01v06 API
Open .eap file of any size 7.2 [+11%] sec 7.2 [+11%] sec
Read full CIM model from .eap file

(~1350 classes, ~5500 [+200] attributes, ~1000 associations, ~80 [+20] dependencies, ~400 [+50] diagrams; [+3% of items])

2.5 min [+25% of time] 4 min [+21% of time]
Read full IEC61850 model from .eap file

(~1300 classes, ~4500 attributes, ~90 associations, ~300 operations, ~370 dependencies, ~200 diagrams)

1.9 min [+27% of time] 3.4 min [+62% of time]
MS Word doc generation 01v06

docgen.saveReopenEvery / duration
IEC61970-301, without Dynamics

(597 tables, 84 figures)
12 / 38.5 min [+7%]
IEC61968-11

(267 tables, 62 figures)
16 / 5.5 min [+10%]
IEC62325-301

(623 tables, 55 figures)
16 / ? min
IEC61850-7-3, with special table formatting

(84 tables, 30 figures)
12 / 2.2 [+50%] min
IEC61850-7-4, with special table formatting

(234 tables, 32 figures; half of tables have 12 more rows than previous models)
12 / 41.2 min [+400%]

Functional changes

New validation rules:

Bug fixes:

Implementation and packaging:

Documentation:


jCleanCim-01v05a, 2011-07-31



jCleanCim-01v05, 2011-07-17


Performance improvements in release 01v05

In release 01v05, we have changed the implementation in many places to optimise for performance. Most significant improvements can be achieved when using the new configuration option: docgen.saveReopenEvery . This option sets the number of tables we generate before we close and reopen the MS Word auto-generated document. The action of closing and reopening the document seems to reset something in the "counting memory" of MS Word, so that we can obtain significant speed improvement. Time to save and reopen file takes in our environment between 1.2-7.7 minutes (maximum is for IEC61970-301 that is the largest document generated). If docgen.saveReopenEvery is too low, it will slow down the generation of the first ~200 tables, when inserting captions is fast. If it is too high, the MS Word "counting memory" when inserting captions grows exponentially and destroys performance.

The config.properties file contains values for docgen.saveReopenEvery which have been proven to be close to the optimum per file tested in our environment, but you can try to slightly increase/decrease that number to see the impact.

The following numbers have been traced from runs on Lenovo ThinkPad T410 (i5 CPU, 3GB memory), 32-bit Windows 7 Enterprise Edition, Office 2007, Enterprise Architect 7.5, Java 1.6.0_24:

Operation Duration

(01v03)
Duration

(01v04, docgen.on)
Duration

(01v05)
Duration

(01v05, docgen.on)
Open .eap file of any size 6.5 sec 6.5 sec 6.5 sec 6.5 sec
Read full CIM model from .eap file

(~1350 classes, ~5300 attributes, ~1000 associations, ~60 dependencies, ~350 diagrams)

4.7 min 5.9 min 2.0 min (no diagram export) 3.3 min (with diagram export)
Read full IEC61850 model from .eap file

(~1000 classes, ~3500 attributes, ~70 associations, ~300 operations, ~330 dependencies, ~130 diagrams)

2.5 min 3.3 min 1.5 min (no diagram export) 2.1 min (with diagram export)
MS Word doc generation of IEC61970-301, with Dynamics

(850 tables, 98 figures, docgen.saveReopenEvery = 12)

- 18 hours! (n/a) 50 min
MS Word doc generation of IEC61970-301, without Dynamics

(597 tables, 84 figures, docgen.saveReopenEvery = 12)

- - (n/a) 36 min
MS Word doc generation of IEC61968-11

(267 tables, 62 figures, docgen.saveReopenEvery = 16)

27 min 29 min (n/a) 5 min
MS Word doc generation of IEC62325-301

(623 tables, 55 figures, docgen.saveReopenEvery = 16)

- 6 hours! (n/a) 31 min
MS Word doc generation of IEC61850-7-3, with special table formatting

(84 tables, 30 figures), docgen.saveReopenEvery = 12)

- 6.2 min (n/a) 1.5 min
MS Word doc generation of IEC61850-7-4, with special table formatting

(234 tables, 32 figures, docgen.saveReopenEvery = 12)

70 min 84 min (n/a) 10.3 min

Changes in release 01v05


jCleanCim-01v04, 2011-03-13



jCleanCim-01v03, 2010-10-03



jCleanCim-01v02, 2010-05-15



JCleanCim-01v01, 2010-02-21



JCleanCim-01v00, 2009-11-11



Built on 2019-12-20T22:05:24

Copyright License

Feedback

Valid XHTML 1.0 Strict