jCleanCim readme
jCleanCim is an open source tool for validation and documentation generation from Enterprise
Architect CIM UML models, and IEC61850 UML models currently under development by UML 61850 task
force of WG10, WG17, WG18 and TC88 JWG25. It is hosted by the CIM Methods & Tools for
Enterprise Integration group on the CIM Users Group web site for the CIM and IEC61850 users
community.
Although this is a Java application, it is unfortunately platform dependent due to the
usage of Enterprise Architect and MS Word, which are available on MS Windows only.
(Old release notes)
Once you unzip a jCleanCim distribution,
doc
directory contains the full documentation. The important parts of the source code are documented
and that documentation is generated as so-called javadoc - namely, a set of web pages
that allow for easy navigation.
- Each distribution contains an up to date set of slides
doc/jCleanCimIntro.pptx
- you may want to start from there.
- Each distribution contains the javadoc in
doc/api/index.html
. The
documentation of the root package org.tanjakostic.jcleancim
(Description section)
is a good starting point.
- Binary distribution contains also the PDF version of that same javadoc, in a file
doc/jCleanCim-[version].pdf
(source distributions contain the Apache ant build script to produce this document, so it is
not packaged in source distributions).
- Source distribution contains the PDF version of the depency graph of the Apache ant
build targets (this is useful for a developer only).
- Finally, each distribution contains the test report, run during the build of the
distribution, under
doc/testReport/index.html
. For most users this is not really
of interest (but keeps our developer spirit in peace :-).
jCleanCim is available in two distributions, depending on how you want to use it.
Even if you have a 64-bit Windows OS, ensure you install a 32-bit Java (JRE: runtime)
or SDK (software development kit) and have it appear on your PATH before the potentially already
installed 64-bit Java, because Enterprise Architect is still a 32-bit application and requires a
32-bit Java. See the commented text in the run.bat script.
Distribution |
User kind |
Prerequisites |
Installing |
Binary distribution jCleanCim-[version]-bin.zip |
Run jCleanCim from the console (cmd.exe). |
- 32-bit Java 7+
runtime (JRE). To verify whether you already have it installed, open the console window
and type
java -version
- Enterprise Architect build 834+ (version 7.1+)
- (optional: to run doc generation) MS Word 2003+
|
Unzip the distribution anywhere on your file system; it will uncompress in its own
directory tagged with the version so there is no danger of overwriting an older
installation. For example, using WinZip, select "Extract to here" command. |
Source distribution jCleanCim-[version]-src.zip |
Run jCleanCim from the console (cmd.exe) or from within eclipse.
Develop and build it with Apache ant or with eclipse.
|
- 32-bit Java 7+
software development kit (SDK). To verify whether you already have it installed, open
the console window and type
javac -version
- Apache ant
1.7.1+. To verify whether you already have it installed, open the console window and
type
ant -version .
- Enterprise Architect build 834+ (version 7.1+)
- (optional, runtime: to run doc generation) MS Word 2003+
- (optional, during ant build: to build javadoc with UML for any distribution, and
to document ant build targets dependencies graph) GraphViz
application. To verify whether you already have it installed, open the console window
and type
dot -version
|
Same as for binary distribution (jCleanCim-[version]-bin.zip).
If used with eclipse, start eclipse SDK and after unzipping, use Import ->Existing
project and browse to the unzipped directory.
Note: This is the most flexible option if you are developing, as you can have the
eclipse project anywhere on your disk (not necessarily in an eclipse workspace). |
discontinued since 02v00
Eclipse project archive jCleanCim-[version]-eclipse.zip |
Same as for Source distribution jCleanCim-[version]-src.zip. |
|
Use Source distribution jCleanCim-[version]-src.zip. |
Note for source distribution (and if you need to create the distribution yourself): Ant
build file contains targets that invoke GraphViz application for creation of graphical elements
for the documentation (UML-enhanced javadoc, and javadoc in pdf format). If you do not have
GraphViz installed on your local system, these targets will be just skipped during the build
(even if you get exceptions from the doclet, the javadoc gets generated, just without the UML
diagrams). However, if you want to produce jCleanCim distributions, you should install GraphViz
in order to have nicer documentation.
To be self-contained, jCleanCim distributions bundle relevant third party open
source/distributable libraries. Java jars are in the project's
lib
directory, and MS Windows dlls are in the
dlls
directory.
The following libraries are packaged with all the distributions of jCleanCim:
- For access to EA model file, we use EA Java API, which in turn depends on a MS Windows
dll. By working directly with EA model file, jCleanCim tries to identify problems at the
source (.eap file), before any XMI or other artefact generation takes place.
- For document generation functionality, we use Jacob library that enables
access from Java to MS Word automation interface, wrapped into a dll for 32bit MS Windows. The
MS Word/Office used is the one on your local machine, and not distributed here.
- For logging, we use Apache's log4j.
- For command line processing, we use Apache's Commons
CLI.
- For string markup processing and some stopwatch functionality, we use Apache's Commons Lang.
The following libraries are packaged only with source distribution of jCleanCim:
- For unit testing, we use JUnit 4. You will need
this only if you run or develop tests, or if you are producing distributions (that include
running tests and producing rest reports).
- For generation of ant target dependencies graph, we use Grand library; it will be ignored if you do
not have GraphViz installed. These are not
used from the jCleanCim source code.
- For enhancement of regular javadoc with the UML class diagrams, we use UmlGraph library; it will be ignored if you do not have GraphViz installed. These are not used from
the jCleanCim source code.
- For pdf generation from the javadoc, we use PDFDoclet application. You will need
this only if you generate the pdf documentation for binary distribution (from an ant target).
The application is otherwise not used from the jCleanCim source code.
Since we talk to EA and to MS Word through their automation APIs, the model building (as a
first step in the application) and the MS Word document generation (if enabled) take time:
- EA automation API implementation unfortunately does not know of bulk CRUD operation, so
for every single item to be returned through the API, they perform an SQL query on the
underlying Access RDBMS (even for items in a collection!). Determinant factor for performance
here are: number of elements (classes, attributes, ...) and the number of diagrams that need
to be saved to file. In release 01v07, we had provided a fully new implementation for reading
the UML model from EA (with option
model.useSql = true
, in that release only). Since
release 01v08, we added one more implementation (see Fast
loading of .eap file) and replaced the model.useSql
boolean option with the one
taking one of three pre-defined string values. In short, if you need to export
XMI for a model release, or diagrams for document generation, ensure you use model.builder=sqlxml
,
otherwise leave the option empty or set it to model.builder=db
. See also hint on fixing ordering errors
.
- MS Word is extremely slow at inserting captions for figures, and in particular for
tables, as well as in populating and formatting tables. As the number of figures/tables grows,
MS Word takes more and more time to insert their captions - similar would happen if you insert
captions by hand in an open Word document: higher the number of captions in the document, more
time MS Word takes to calculate the number for the caption (and it is impossible to disable
this automatic calculation if we want to create tables of figures/tables). Since
release 01v05, we provide a configuration option
docgen.saveReopenEvery
that
you should definitely use to speed up MS Word document generation
. Default value is 12, but you should play with your document to find out whether higher value
would make it faster. See also discussion on
this option and its resulting performance improvement.
Java processing - for validation, statistics calculation and documentation collection from
in-memory model to pass to the actual writer (s), as well as XML document generation for
web-access - takes a couple of seconds for all the models and scenarios tested.
Here items thay may be considered as issues (but will likely not be addressed soon) and
performance-related advice, so please take them into account when running jCleanCim:
- If you run only to validate the model or generated XML or Word doc without
diagrams, you should use
model.builder=db
as the fastest way; notice that no
digrams can ever get exported with this builder.
With any other builder option, and when docgen.on=true
, diagrams get exported from
the EA model (in order to be used in the generated doc), while if docgen.on=false
(or empty, or absent), document generation is disabled and we know that we don't need
diagrams, so they don't get exported at all. Not exporting diagrams saves a lot of time:
between 300-500 ms per normative diagram - for 100 normative diagrams, you save at least half
a minute to read the model.
- EA ordering errors. If you use
model.builder=db
or model.builder=sqlxml
in 01v08, or model.useSql=true
in 01v07
(which you should, because they are orders of magnitude faster), you may see logged ERROR (+++
EA ordering error) for several UML elements (diagrams, packages, classes, etc.). This comes
from the fact that EA internal storage for some reason does not always keep up to date the
order of items in a container (we have seen this already with attributes order before). When
using API, we just follow the order of elements as returned by the EA API. However, when
processing SQL query results or reading the tables directly, we may find those position
indexes totally uninitialised (that's how they are stored in the repository). To ensure the
order gets preserved, just open the model and manually move an item from the indicated list up
and down - this should trigger the EA internal update mechanism - and then you're set as the
error should disappear on the next run.
- If for some reason you're still using
model.builder=japi
for reading from
EA file:
- Remove any baselines. They grow the size of data in the underlying RDBMS and every
SQL query that EA does (unfortunately: 1 for each single
attribute/association/class/package/tagged value/constraint/dependency) will take longer
on a larger dataset.
- Compact EAP file. From within EA, run regularly Tools - Manage .EAP file - Compact
.EAP file. There is no effect if this is executed from EA that you've opened with your own
model (that is why this is not feasible programmatically either), so you must open EA with
an arbitrary, another model (or just launch EA alone), then select your file to compact.
- When running MS Word document generation:
- (since 01v08) If you can, save your template as native Office 2007/2010/2013
document (.docx), without compatibility enabled. This will run much faster, because in
this case only, it is possible to programmatically disable field updates.
- This will run correctly only on an English version of MS Office (or on a local
language installation plus the language support package for English). Reason: We
create MS Word documents primarily for IEC, and these need to be in English; supporting
various languages in MS Word (for e.g. style names) would be too much work at this moment.
If you have a patch that can generically support MS Word installation in any local
language, please submit it.
- Ensure to update all fields in the MS Word template before starting document
generation with change tracking turned OFF , in particular if you have
added/removed tables/figures and their captions. MS Word is known to screw up caption
numbers (read: identifiers) when you edit them with change tracking enabled.
- Disable automatic spell checking in the styles 'PARAGRAPH', 'TABLE-cell' and
'Normal'.
- On our developer Windows 7 machine, we have noticed that for some special
formatting of diagrams (with resizing or such), the process PrintSpooler (the Windows
printing service) kicks in immediately after EA repository gets opened. This manifests at
building Domain package from the sample EA repository, when it seems like never-ending.
When stopping the PrintSpoolerService (Computer/Manage/Services and
Applications/Services/PrintSpooler/Stop) at that point, there are about 20 popups from EA
process reporting on some font problem (that is how we concluded it is about some
interaction of EA with the OS). If this manifests on your machine, the easiest is to
temporarily stop PrintSpooler service while running jCleanCim. Note that this has never
happened on Windows XP, only on Windows 7.
Starting with release 02v00, jCleanCim has been licensed under the terms of GNU LGPLv3 license and includes a modified copyright.
The copyright as well as a reference to the license for this software is available at the
download site, and is included in every distribution and in every java source file.
Have a look here for a relatively
accessible comparison of licenses.
This software has been developed in my free time, with the full knowledge and approval of my employer.
Licensing changes
From this release, the license terms have changed, see disclaimer. There
should be no big impact if you were using it the same way as I did :-)
Distribution changes
Producing the distribution "Eclipse project archive jCleanCim-[version]-eclipse.zip" has
been discontinued, because it's a manual and time consuming process. The same functionality is
available by using the "Source distribution jCleanCim-[version]-src.zip" - by importing simply
the eclipse project file from un unzipped directory.
New features
This is a release with incremental changes and no "real" new feature.
Performance improvements in release 02v00
None.
Potential backwards compatibility breaking changes for the application user
- validation: Renamed rule ClassesWithSuperclassesFromUnallowedPackage to
ClassesWithSuperclassesFromUnallowedOwner to better reflect the intent (and also to
better align with the new rule applying to attributes of a class - see below).
- all: If you have been using source code or the .jar as a library, the root package name
has changed in this release.
Potential forwards incompatibility for the application user: old jCleanCim with new EA
- jCleanCim releases older than 02v00 may have a problem when closing .eap file with
Enterprise Architect 12 and higher. The reason is that EA12 Java API changed (they removed a
method that was used in the code just before closing the .eap file; and jCleanCim-02v00
replaces that method call appropriately). In our test installation (EA12 + jCleanCim-01v10),
there was a pop-up window telling that EA application stopped running and clicking on "Close"
application did a smooth shut down of EA - then the run continues normally.
Other functional changes
- builder.ea: Improved identification and signaling of the problematic entries in
messages "+++ EA ordering problem for ..." (when reading EA model); e.g. "+++ class
ExciterType: pos = 22 DUPLICATE". Moving any one of the entries ending with DUPLICATE
should initiate EA's renumbering and solve the problem in that given scope (e.g., package for
ordering problem with its diagrams or classes, class for ordering problem with its
attributes).
- CIM-specific:
- IEC61850-specific (as usual):
- config61850.properties: adjusted some changed package names from UML and added
ACSITypesEnums.
- model, docgen: added support for one more stereotype (structured) and the
corresponding non-meta-model type (with prefix S_) - this is at present a kludge to obtain
in the Word what the editors want to see.
- docgen.writer.xml: adapted schema to accept structured type in 7-2 (credit to Chris
Frei).
- model, collector: extended usage of 'scl:emptyValue' tag not only for enum literal,
but also for any IEC61850 attribute (specifically needed for Unit.multiplier), to simply
override the default value ('none') with an empty string. Good luck, IEC61850 model
managers!
- collector: on request of 7-2/7-3 editors, replaced the table column text "M/O/C"
with 'PresCond".
- collector: on request of 7-2/7-3 editors, simplified printing of enumerated types
in 7-2 and 7-3: instead of "ENUMERATED (XyzKind)" or "CODED ENUM (XyzKind)", now printing
simply "XyzKind".
- collector: implemented support for SCL enumeration printing to use the package name
in the heading (to allow printing a couple of packages one after another); in older
version, we were always printing a single package in an annex; now we can print any number
of enum packages (as SCL enums) anywhere.
- model, collector; on request of 7-3 editors, added "datId = " in the table title
for all data attribute types (ConstructedDAs).
- collector: on request of 7-4 editors, change ‘enumeration literal’ in table
headings / captions to “enumeration item�.
- collector: on request of 7-4 editors, don’t output the package name in table
captions.
Validation rules:
- util (validation): generalised CimAttributesNameShouldBeSingular to accept anything
ending with 'ous' and 'ius' as singular (e.g., instantaneous, previous, anonymous, radius);
with 'bias'; with 'rans' (for Trans(ient) and Subtrans(ient)).
- New common / general validation rules:
- AttributesWithTypeFromUnallowedPackage (for associations, this is already checked
with AssociationsWithWrongSource)
- New CIM-specific rules:
- Removed CIM-specific rules:
- EnumLiteralsWithInitValue, because CIM also now allows codes for literals.
- New IEC61850-specific rules:
Bug fixes:
- stats: distinguishing "61850" enum literal from "any" enum literal.
- model (61850): distinguishing the case where presence condition literal with argument
is used as-is, without instantiating the argument with a value (misuse of e.g. AtLeastOne(n)
instead of AtLeastOne(1)) - credits to Chris Frei for pointing out the bug.
Known issues / limitations:
Implementation and packaging:
- upgraded IECDomain.xsd to 0.13.
- upgraded jackcess library to 2.1.0.
- upgraded EA dll and jar to version 12 (from EA build 1212).
- removed docx4j and dependencies, will try to use Apache POI instead at some later date.
- added some more unit tests.
- included oldReleaseNotes.html for distribution in the build script.
Documentation:
- updated the presentation.
Built on 2015-05-19T13:26:17
License
Feedback