Sometimes it is useful to construct a working copy that is made out of a number of different checkouts. For example, you may want different subdirectories to come from different locations in a repository or perhaps from different repositories altogether. You could certainly set up such a scenario by hand—using svn checkout to create the sort of nested working copy structure you are trying to achieve. But if this layout is important for everyone who uses your repository, every other user will need to perform the same checkout operations that you did.
Fortunately, Subversion provides support for
externals definitions. An externals
definition is a mapping of a local directory to the
URL—and ideally a particular revision—of a versioned
directory. In Subversion, you declare externals definitions in
groups using the
svn:externals property. You
can create or modify this property using svn
propset or svn propedit (see the section called “Manipulating Properties”). It can be set on any
versioned directory, and its value describes both the external
repository location and the client-side directory to which that
location should be checked out.
The convenience of the
property is that once it is set on a versioned directory,
everyone who checks out a working copy with that directory also
gets the benefit of the externals definition. In other words,
once one person has made the effort to define the nested working
copy structure, no one else has to bother—Subversion will,
after checking out the original working copy, automatically also
check out the external working copies.
The relative target subdirectories of externals definitions must not already exist on your or other users' systems—Subversion will create them when it checks out the external working copy.
You also get in the externals definition design all the
regular benefits of Subversion properties. The definitions are
versioned. If you need to change an externals definition, you
can do so using the regular property modification subcommands.
When you commit a change to the
property, Subversion will synchronize the checked-out items
against the changed externals definition when you next run
svn update. The same thing will happen when
others update their working copies and receive your changes to
the externals definition.
svn:externals property has
a multiline value, we strongly recommend that you use
svn propedit instead of svn
Subversion releases prior to 1.5 honor an externals definition format that is a multiline table of subdirectories (relative to the versioned directory on which the property is set), optional revision flags, and fully qualified, absolute Subversion repository URLs. An example of this might look as follows:
$ svn propget svn:externals calc third-party/sounds http://svn.example.com/repos/sounds third-party/skins -r148 http://svn.example.com/skinproj third-party/skins/toolkit -r21 http://svn.example.com/skin-maker
When someone checks out a working copy of the
calc directory referred to in the previous
example, Subversion also continues to check out the items found
in its externals definition.
$ svn checkout http://svn.example.com/repos/calc A calc A calc/Makefile A calc/integer.c A calc/button.c Checked out revision 148. Fetching external item into calc/third-party/sounds A calc/third-party/sounds/ding.ogg A calc/third-party/sounds/dong.ogg A calc/third-party/sounds/clang.ogg … A calc/third-party/sounds/bang.ogg A calc/third-party/sounds/twang.ogg Checked out revision 14. Fetching external item into calc/third-party/skins …
As of Subversion 1.5, though, a new format of the
svn:externals property is supported.
Externals definitions are still multiline, but the order and
format of the various pieces of information have changed. The
new syntax more closely mimics the order of arguments you might
pass to svn checkout: the optional revision
flags come first, then the external Subversion repository URL,
and finally the relative local subdirectory. Notice, though,
that this time we didn't say “fully qualified, absolute
Subversion repository URLs.” That's because the new
format supports relative URLs and URLs that carry peg revisions.
The previous example of an externals definition might, in
Subversion 1.5, look like the following:
$ svn propget svn:externals calc http://svn.example.com/repos/sounds third-party/sounds -r148 http://svn.example.com/skinproj third-party/skins -r21 http://svn.example.com/skin-maker third-party/skins/toolkit
Or, making use of the peg revision syntax (which we describe in detail in the section called “Peg and Operative Revisions”), it might appear as:
$ svn propget svn:externals calc http://svn.example.com/repos/sounds third-party/sounds http://svn.example.com/skinproj@148 third-party/skins http://svn.example.com/skin-maker@21 third-party/skins/toolkit
You should seriously consider using explicit revision numbers in all of your externals definitions. Doing so means that you get to decide when to pull down a different snapshot of external information, and exactly which snapshot to pull. Besides avoiding the surprise of getting changes to third-party repositories that you might not have any control over, using explicit revision numbers also means that as you backdate your working copy to a previous revision, your externals definitions will also revert to the way they looked in that previous revision, which in turn means that the external working copies will be updated to match the way they looked back when your repository was at that previous revision. For software projects, this could be the difference between a successful and a failed build of an older snapshot of your complex codebase.
For most repositories, these three ways of formatting the
externals definitions have the same ultimate effect. They all
bring the same benefits. Unfortunately, they all bring the same
annoyances, too. Since the definitions shown use absolute URLs,
moving or copying a directory to which they are attached will
not affect what gets checked out as an external (though the
relative local target subdirectory will, of course, move with the
renamed directory). This can be confusing—even
frustrating—in certain situations. For example, say you
have a top-level directory named
my-project, and you've created an externals
definition on one of its subdirectories
my-project/some-dir) that tracks the
latest revision of another of its subdirectories
$ svn checkout http://svn.example.com/projects . A my-project A my-project/some-dir A my-project/external-dir … Fetching external item into 'my-project/some-dir/subdir' Checked out external at revision 11. Checked out revision 11. $ svn propget svn:externals my-project/some-dir subdir http://svn.example.com/projects/my-project/external-dir $
Now you use svn move to rename the
my-project directory. At this point, your
externals definition will still refer to a path under the
my-project directory, even though that
directory no longer exists.
$ svn move -q my-project renamed-project $ svn commit -m "Rename my-project to renamed-project." Deleting my-project Adding renamed-project Committed revision 12. $ svn update Updating '.': svn: warning: W200000: Error handling externals definition for 'renamed-projec t/some-dir/subdir': svn: warning: W170000: URL 'http://svn.example.com/projects/my-project/externa l-dir' at revision 12 doesn't exist At revision 12. svn: E205011: Failure occurred processing one or more externals definitions $
Also, absolute URLs can cause problems with repositories
that are available via multiple URL schemes. For example, if
your Subversion server is configured to allow everyone to check
out the repository over
https://, but only allow commits to come in
https://, you have an interesting problem
on your hands. If your externals definitions use the
http:// form of the repository URLs, you
won't be able to commit anything from the working copies created
by those externals. On the other hand, if they use the
https:// form of the URLs, anyone who might
be checking out via
http:// because his
client doesn't support
https:// will be
unable to fetch the external items. Be aware, too, that if you
need to reparent your working copy (using svn
relocate), externals definitions will
not also be reparented.
Subversion 1.5 takes a huge step in relieving these frustrations. As mentioned earlier, the URLs used in the new externals definition format can be relative, and Subversion provides syntax magic for specifying multiple flavors of URL relativity.
Relative to the URL of the directory on which the
svn:externalsproperty is set
Relative to the root of the repository in which the
svn:externalsproperty is versioned
Relative to the scheme of the URL of the directory on which the
svn:externalsproperty is set
Relative to the root URL of the server on which the
svn:externalsproperty is versioned
So, looking a fourth time at our previous externals definition example, and making use of the new absolute URL syntax in various ways, we might now see:
$ svn propget svn:externals calc ^/sounds third-party/sounds /skinproj@148 third-party/skins //svn.example.com/skin-maker@21 third-party/skins/toolkit $
Subversion 1.6 brought two more improvements to externals
definitions. First, it added a quoting and escape mechanism to
the syntax so that the path of the external working copy may
contain whitespace. This was previously problematic, of course,
because whitespace is used to delimit the fields in an externals
definition. Now you need only wrap such a path specification in
") characters or
escape the problematic characters in the path with a backslash
\) character. Of course, if you have spaces
in the URL portion of the external
definition, you should use the standard URI-encoding mechanism
to represent those.
$ svn propget svn:externals paint http://svn.thirdparty.com/repos/My%20Project "My Project" http://svn.thirdparty.com/repos/%22Quotes%20Too%22 \"Quotes\ Too\" $
Subversion 1.6 also introduced support for external definitions for files. File externals are configured just like externals for directories and appear as a versioned file in the working copy.
For example, let's say you had the file
/trunk/bikeshed/blue.html in your repository,
and you wanted this file, as it appeared in revision 40,
to appear in your working copy of
The externals definition required to achieve this should look familiar by now:
$ svn propget svn:externals www/ ^/trunk/bikeshed/blue.html@40 green.html $ svn update Updating '.': Fetching external item into 'www' E www/green.html Updated external to revision 40. Update to revision 103. $ svn status X www/green.html $
As you can see in the previous output, Subversion denotes file
externals with the letter
E when they are
fetched into the working copy, and with the letter
X when showing the working copy status.
While directory externals can place the external directory at any depth, and any missing intermediate directories will be created, file externals must be placed into a working copy that is already checked out.
When examining the file external with svn info, you can see the URL and revision the external is coming from:
$ svn info www/green.html Path: www/green.html Name: green.html Working Copy Root Path: /home/harry/projects/my-project URL: http://svn.example.com/projects/my-project/trunk/bikeshed/blue.html Repository Root: http://svn.example.com/projects/my-project Repository UUID: b2a368dc-7564-11de-bb2b-113435390e17 Revision: 40 Node kind: file Schedule: normal Last Changed Author: harry Last Changed Rev: 40 Last Changed Date: 2009-07-20 20:38:20 +0100 (Mon, 20 Jul 2009) Text Last Updated: 2009-07-20 23:22:36 +0100 (Mon, 20 Jul 2009) Checksum: 01a58b04617b92492d99662c3837b33b $
Because file externals appear in the working copy as versioned files, they can be modified and even committed if they reference a file at the HEAD revision. The committed changes will then appear in the external as well as the file referenced by the external. However, in our example, we pinned the external to an older revision, so attempting to commit the external fails:
$ svn status M X www/green.html $ svn commit -m "change the color" www/green.html Sending www/green.html svn: E155011: Commit failed (details follow): svn: E155011: File '/trunk/bikeshed/blue.html' is out of date $
Keep this in mind when defining file externals.
If you need the external to refer to a certain revision
of a file you will not be able to modify the external.
If you want to be able to modify the external, you cannot
specify a revision other than the
revision, which is implied if no revision is specified.
Unfortunately, the support which exists for externals definitions
in Subversion remains less than ideal. Both file and directory
externals have shortcomings. For either type of external, the
local subdirectory part of the definition cannot contain
.. parent directory indicators (such as
../../skins/myskin). File externals cannot
refer to files from other repositories. A file external's URL
must always be in the same repository as the URL that the file
external will be inserted into. Also, file externals cannot be
moved or deleted. The
must be modified instead. However, file externals can be copied.
Perhaps most disappointingly, the working copies created via the
externals definition support are still disconnected from the primary
working copy (on whose versioned directories the
svn:externals property was actually set).
And Subversion still truly operates only on nondisjoint working
copies. So, for example, if you want to commit changes that
you've made in one or more of those external working copies, you
must run svn commit explicitly on those
working copies—committing on the primary working copy will
not recurse into any external ones.
We've already mentioned some of the additional shortcomings
of the old
svn:externals format and how the
newer Subversion 1.5 format improves upon it. But be careful
when making use of the new format that you don't inadvertently
introduce new problems. For example, while the latest clients
will continue to recognize and support the original externals
definition format, pre-1.5 clients will not
be able to correctly parse the new format. If you change all
your externals definitions to the newer format, you effectively
force everyone who uses those externals to upgrade their
Subversion clients to a version that can parse them. Also, be
careful to avoid naively relocating
of the definition—the older format uses that revision as a
peg revision, but the newer format uses it as an operative
revision (with a peg revision of
otherwise specified; see the section called “Peg and Operative Revisions”
for a full explanation of the distinction here).
External working copies are still completely
self-sufficient working copies. You can operate directly on
them as you would any other working copy. This can be a handy
feature, allowing you to examine an external working copy
independently of any primary working copy
svn:externals property caused its
instantiation. Be careful, though, that you don't
inadvertently modify your external working copy in subtle ways
that cause problems. For example, while an externals
definition might specify that the external working copy should
be held at a particular revision number, if you
run svn update directly on the external
working copy, Subversion will oblige, and now your external
working copy is out of sync with its declaration in the
primary working copy. Using svn switch to
directly switch the external working copy (or some portion
thereof) to another URL could cause similar problems if the
contents of the primary working copy are expecting particular
contents in the external content.
Besides the svn checkout, svn
update, svn switch, and
svn export commands which actually manage the
disjoint (or disconnected) subdirectories
into which externals are checked out, the svn
status command also recognizes externals definitions.
It displays a status code of
X for the
disjoint external subdirectories, and then recurses into those
subdirectories to display the status of the external items
themselves. You can pass the
--ignore-externals option to any of these
subcommands to disable externals definition processing.