$Id: jpackage-1.5-policy.xhtml,v 1.2 2005/09/17 07:06:26 david Exp $
Abstract
This document describes the packaging policy followed by the JPackage cross-distribution RPM Java™ packaging project.
Table of Contents
List of Examples
Clean Java packaging has historically been a daunting task. Lack of any standard addressing the physical location of files on the system combined with the common use of licensing terms that only allow free redistribution of key components as a part of a greater ensemble has let to the systematic release of self-sufficient applications with built-in copies of external components.
As a consequence applications are only tested with the versions of the components they bundle, a complete Java system suffers from endless duplication of the same modules, and integrating multiple parts can be a nightmare since they are bound to depend on the same elements - only with different and subtly incompatible versions[1]. Any security or compatibility[2] upgrade must be performed for each of those duplicated elements.
This problem is compounded by the current practice of folding extensions proper in the JVM itself after a while ; an element that could safely be embedded in a application will suddenly conflict with a JVM part and cause subtle failures.
It is not surprising then that complex Java systems tend to fossilize very quickly, with the cost of maintaining dependencies current growing too high so fast people basically give up on it.
This situation is incompatible with your typical fast-evolving Linux platform. To attain its aim of user- and administrator-friendly rpm packaging of Java applications the JPackage Project had to evolve its own system infrastructure and strict packaging rules.
Whenever possible a module shall be broken out in its constituting elements. A given element will have a single name and location on the system. Only a single version of this element shall be installed. Applications sharing the same dependencies shall depend on external packages, not built-in versions.
The user shall be able to choose between the different available implementations of the same element, especially when their licensing terms are not equivalent or their level of completeness unequal. Whenever possible it should be possible to install them in parallel, with the user able to select each of them specifically or let the system choose the best one[3].
Packages should work on most major rpm-based distributions. This means following standards like the Linux Standard Base, the Filesystem Hierarchy Standard, or freedesktop.org specifications.
Above all usage of distribution-specific facilities[4] is forbidden.
Note this is a pragmatic reading of the standards. Facilities not specified but commonly found are readily used by the project[5] and distribution-specific adaptations are tolerated provided they do not interfere with common usage or a generic version is also available.
Table of Contents
Packages shall be named after the common name of their origin project in small caps. When a package provides an extension that was folded at some point in the Java standard, the -ext (as in external) suffix shall be appended to distinguish between the package name and the extension name. Both the JVMs including this extension and the standalone extension package shall then have the original name as a virtual Provides.
JVM packages shall be named java-standard_version-vendor. Original on-site names are not followed since practice varies wildly from vendor to vendor and version to version.
If a package provides a single jar it shall have the same name as the package itself, with appended product version.
jaf-1.0.2.jar
A non-versioned symbolic link pointing on the original file shall also be provided.
jaf-1.0.2.jar jaf.jar → jaf-1.0.2.jar
If the project name and the commonly used jar name differ a symbolic link with the usual name shall also be provided.
If the package provides several jars their usual names will be used.
ant-1.5.3.jar ant.jar → ant-1.5.3.jar ant-optional-1.5.3.jar ant-optional.jar → ant-optional-1.5.3.jar
If the number of provided jars exceeds two or if they were at a point provided in a single monolithic jar the files should be placed in a sub-directory named after the package (as for a single jar).
If a project offers the choice of packaging it as a single monolithic jar or several ones, the split packaging will be preferred.
Table of Contents
Core directory structure is provided by the jpackage-utils package. It consists of:
The %{_javadir} rpm macro defines the main jar repository. Historically it was the sole directory used by the 1.0 JPackage distribution, before packaging constraints forced a more complex system. It usually expands into /usr/share/java.
All jar files and directories of jar files that do not depend on a particular Java standard version or JNI shall be installed in %{_javadir}.
From %{_javadir} we derive %{_javadir}-ext. All jar files and directories of jar files that depend on a particular Java standard version but not JNI shall be installed in %{_javadir}-ext.
We also derive the %{_javadir}-x.y.z directories. They contain symlinks to files or directories of %{_javadir}-ext for all elements that are valid for the x.y.z Java standard version. Since an implementation is usually valid for a range of Java standard iterations, a file or directory in %{_javadir}-ext will usually have several symbolic links pointing to it. Also of note are the non-versioned symlinks : for two jars named foo13.jar and foo14.jar, the foo.jar symlink will point to foo13.jar in %{_javadir}-1.3.0 and %{_javadir}-1.3.1, and foo14.jar in %{_javadir}-1.4.0, %{_javadir}-1.4.1 and %{_javadir}-1.4.2.
Unfortunately the Java standard has been known to change enough between minor versions that we must take into account the full version and distinguish between %{_javadir}-1.4.0 and %{_javadir}-1.4.1.
Example 3.1. Java standard version-specific repositories usage
/usr/share/java-ext/jsse /usr/share/java-ext/jsse/jcert-1.0.3.01.jar /usr/share/java-ext/jsse/jcert.jar → jcert-1.0.3.01.jar /usr/share/java-ext/jsse/jnet-1.0.3.01.jar /usr/share/java-ext/jsse/jnet.jar → jnet-1.0.3.01.jar /usr/share/java-ext/jsse/jsse-1.0.3.01.jar /usr/share/java-ext/jsse/jsse.jar → jsse-1.0.3.01.jar /usr/share/java-1.3.0/jsse → /usr/share/java-ext/jsse /usr/share/java-1.3.1/jsse → /usr/share/java-ext/jsse
The %{_jnidir} rpm macro defines the main JNI jar repository. Like %{_javadir} it is declined in -ext and -x.y.z variants. It follows exactly the same rules as the %{_javadir}-derived tree structure, except that it hosts jars that use JNI.
%{_jnidir} usually expands into /usr/lib/java.
The %{_jvmdir} rpm macro defines the root directory under which the different system JVMs are installed. It usually expands into /usr/lib/jvm.
From %{_jvmdir} we derive %{_jvmdir}-exports. Each subdirectory of %{_jvmdir} must have a corresponding one in %{_jvmdir}-exports. It is used to register Java extensions bundled with the SDK or RE with symbolic links pointing inside the JVM structure in %{_jvmdir}.
The symbolic links shall point to the actual in-JVM jar file providing the extension (though that is not strictly necessary for the system to work), be available in versioned and non-versioned versions and follow general naming rules.
The %{_jvmdir}-private directory contains files that are private and internal to JVMs, but for some reason need to be relocated from their normal location in the JVM dirs. No scripts should reference these files. The contents below this directory are laid out so that there are appropriately versioned directories depending on their purpose, and the actual files are below those.
One example of private files are the Java Cryptography Extension (JCE) policy files; the ones that come with various 1.4.x JVMs are limited in strength and the vendors supply unlimited strength jurisdiction policy files as separate downloads. Typically these files are Java standard version and vendor specific (eg. java-1.4.2-sun).
Example 3.2. JCE policy file layout for Sun's J2SE 1.4.2
/usr/lib/jvm-private/java-1.4.2-sun /usr/lib/jvm-private/java-1.4.2-sun/jce /usr/lib/jvm-private/java-1.4.2-sun/jce/vanilla /usr/lib/jvm-private/java-1.4.2-sun/jce/vanilla/US_export_policy.jar /usr/lib/jvm-private/java-1.4.2-sun/jce/vanilla/local_policy.jar /usr/lib/jvm-private/java-1.4.2-sun/jce/unlimited /usr/lib/jvm-private/java-1.4.2-sun/jce/unlimited/US_export_policy.jar /usr/lib/jvm-private/java-1.4.2-sun/jce/unlimited/local_policy.jar
The various versions of JCE policy jars are further controlled by the alternatives(8) system, using a link pointing to the corresponding jars in the JVM's jre/lib/security directory, with the unlimited versions having higher priority than the ones shipped with the JVM (vanilla).
%{_sysconfdir}/java hosts the general configuration files related to the java subsystem, mainly java.conf.
This is the root of all installed javadoc documentation. It's location and intended usage are debated right now.
If an application needs private directories they should be laid out on the system following Filesystem Hierarchy Standard rules.
If an application needs to simulate a single-root private tree like on other operating systems (when the FHS requires the subdirectories to be installed on different parts of the system), %{_datadir}/appname shall be used as application home with symbolic links pointing to the real locations of subdirectories on the system. Of course it is better to fix the application to understand proper split file layout and avoid the symbolic link indirection altogether.
Example 3.3. FHS and homedir-centered file layout
/usr/share/tomcat4 /usr/share/tomcat4/bin /usr/share/tomcat4/common → /var/lib/tomcat4/common /usr/share/tomcat4/conf → /etc/tomcat4 /usr/share/tomcat4/logs → /var/log/tomcat4 /usr/share/tomcat4/server → /var/lib/tomcat4/server /usr/share/tomcat4/shared → /var/lib/tomcat4/shared /usr/share/tomcat4/temp → /var/cache/tomcat4/temp /usr/share/tomcat4/webapps → /var/lib/tomcat4/webapps /usr/share/tomcat4/work → /var/cache/tomcat4/work
Gratuitous complexity is eschewed. If a directory will always have only a single subdirectory, collapse it[6].
If an application uses classpath elements that are not jar files they should be installed in a private application directory.
One may encounter other types of archives used in classpaths. If the application has no special need of them being a particular file type, they should be converted to jars and used the usual way. This is particularly true of zip files that were in wide usage at a time but have been deprecated since.
If an application provides private jars files it will always be the only user of, they may be installed in private application directories.
This implies generic routines can not be used to build the application classpath, so this part must be handled by the application itself or its packager[7]. However when an application can and will read its private jar repository, we do provide the means to maintain the repository parts that are shared with other applications.
Table of Contents
We assume the correct environment variables have already been set, ie at least $JAVA_HOME to select a JVM in %{_jvmdir}, and eventually the relevant $JAVACMD, $LD_ASSUME_KERNEL, $LANG, and $JAVA_COMPILER variables.
The current implementation is not sane and carries a lot of legacy garbage. A clean implementation would be to read these values in ~/.apprc, falling back on /etc/app.conf, the user environment, ~/.java and /etc/java/java.conf. If $JAVA_HOME was still undefined then, use/usr/lib/jvm/java as default.
Unfortunately we are not doing so right now.
When queried for foo/bar-x.y we search for the foo/bar-x.y.jar jar file, then the foo/bar-x.y jar directory in the following locations:
%{_jvmdir}-exports/name . The shadow of the JVM defined by $JAVA_HOME=%{_jvmdir}/name. This is the JVM-specific repository where we register the Java extensions it includes.
%{_jnidir}-java_version . Where java_version is the JVM standard Java compliance level as deduced from
[bob@sys dir]$ $JAVACMD -version
%{_javadir}-java_version . The java-version specific non-JNI jar repository.
%{_jnidir} . The generic JNI jar repository.
%{_javadir} . The generic jar repository.
If we didn't find anything, the search is repeated for foo/bar.jar, foo/bar, foo.jar then at last the foo directory. Note the scan is performed for jar/directory pairs ie a subdirectory located in a more specific repository will always have precedence on a jar with the same name located in a less specific repository.
find-jar {element}
The find-classpath command tests the resolution of a given element. If successful it will return a jar file or a directory. It is solely intended for testing resolution and should not be used in scripts. Even for single-element classpath building the following command is preferred, since single-element searches can produce multiple file results when resolving into a directory.
build-classpath {element...}
The build-classpath command builds a standard classpath following the general resolution rules. It takes a list of elements as arguments. If an element resolves in a directory, all its children jar files are included in the classpath.
Example 4.2. build-classpath output
[bob@sys dir]$ export JAVA_HOME=/usr/lib/jvm/java-1.3.1-blackdown [bob@sys dir]$ build-classpath jsse javamail/mailapi jaxp_parser_impl /usr/share/java-1.3.1/jsse/jcert-1.0.3.01.jar:/usr/share/java-1.3.1/jsse/jnet-1.0.3.01.jar:/usr/share/java-1.3.1/jsse/jsse-1.0.3.01.jar:/usr/share/java/javamail/mailapi.jar:/usr/share/java/jaxp_parser_impl.jar [bob@sys dir]$ export JAVA_HOME=/usr/lib/jvm/java-1.4.1-sun [bob@sys dir]$ build-classpath jsse javamail/mailapi jaxp_parser_impl /usr/lib/jvm-exports/java-1.4.1-sun/jsse.jar:/usr/share/java/javamail/mailapi.jar:/usr/share/java/jaxp_parser_impl.jar
Like the following command it will fail, return an non-zero error code and write to the error console if one or more of the requested elements were not found. Therefore for a classpath composed of required and optional parts recommended building practice is the following:
build-jar-repository [[-s] | [--soft] | [--symbolic] | [-h] | [--hard] | [-c] | [--copy]] [[-p] | [--preserve-naming]] {directory} {element...}
The build-jar-repository command constructs a flat directory of jar file symlinks following the same rules. It takes a directory name followed by a list of elements as arguments. It will try to create a [foo][bar]xxx.jar set of symbolic links in the target directory for each foo/bar requested element[8]. The special symlink naming makes it possible to recognize they were created by this command and is required for the next command. One can optionally specify if created files must be hardlinks, softlinks or file copies. The default is symbolic links, and the switch should only be needed for very broken software.
build-jar-repository can be safely used on the same directory several times. Previous symlinks won't be removed. If one wants to reset the directory the symbolic links must be cleaned up manually.
If you just want to populate a static directory with jar links or copies you can use the --preserve-naming build-jar-repository argument. This will create files with a name closer to the original ones. Note however this will inhibit any future repository refresh using the next command. Do not use this switch unless you know what you are doing.
--preserve-naming implies --copy unless specified otherwise.
Example 4.4. build-jar-repository results
[bob@sys dir]$ export JAVA_HOME=/usr/lib/jvm/java-1.3.1-blackdown [bob@sys dir]$ build-jar-repository lib jsse javamail/mailapi [bob@sys dir]$ build-jar-repository lib jaxp_parser_impl [bob@sys dir]$ tree lib [nim@rousalka nim]$ tree lib lib |-- [javamail][mailapi].jar -> /usr/share/java/javamail/mailapi.jar |-- [jaxp_parser_impl].jar -> /usr/share/java/jaxp_parser_impl.jar |-- [jsse]jcert-1.0.3.01.jar -> /usr/share/java-1.3.1/jsse/jcert-1.0.3.01.jar |-- [jsse]jcert.jar -> /usr/share/java-1.3.1/jsse/jcert.jar |-- [jsse]jnet-1.0.3.01.jar -> /usr/share/java-1.3.1/jsse/jnet-1.0.3.01.jar |-- [jsse]jnet.jar -> /usr/share/java-1.3.1/jsse/jnet.jar |-- [jsse]jsse-1.0.3.01.jar -> /usr/share/java-1.3.1/jsse/jsse-1.0.3.01.jar `-- [jsse]jsse.jar -> /usr/share/java-1.3.1/jsse/jsse.jar
rebuild-jar-repository [[-s] | [--soft] | [--symbolic] | [-h] | [--hard] | [-c] | [--copy]] {directory}
The rebuild-jar-repository command refreshes a jar repository created using build-jar-repository. It takes a directory name as argument, and is intended to be run after $JAVA_HOME was changed to insure all symlinked jar files are still compatible with the new JVM. The same optional arguments as build-jar-repository can be used to specify the kind of link to use.
The very specific naming of the symbolic links used by build-jar-repository and rebuild-jar-repository makes it possible to mix automated links with manual links and normal files in directories handled by those commands. The scripts will only change their own links.
Example 4.5. rebuild-jar-repository results
[bob@sys dir]$ export export JAVA_HOME=/usr/lib/jvm/java-1.4.1-sun [bob@sys dir]$ rebuild-jar-repository lib [bob@sys dir]$ tree lib lib |-- [javamail][mailapi].jar -> /usr/share/java/javamail/mailapi.jar |-- [jaxp_parser_impl].jar -> /usr/share/java/jaxp_parser_impl.jar `-- [jsse].jar -> /usr/lib/jvm-exports/java-1.4.1-sun/jsse.jar
rebuild-jar-repository will still create a symbolic link unlike build-jar-repository. This choice was made to avoid corrupting the element list when selecting an incomplete java system[9]. The created symbolic link points to a bogus location and should always be "broken".