Michael Johnson and I spent a good hunk of last week finishing off derived packages. It finally got pushed into the Conary 1.1 tree Friday afternoon and should make it into a released Conary soon. Since I think this is one of the most interesting things to make it into Conary in a while, let me talk about what it does.
Derived packages are meant to answer the “I just need to change foo; why do I need to recompile the whole thing?” question. That could be “I just want to remove some of these scsi modules from the kernel” or “how do I change /etc/httpd/httpd.conf?”. Our standard answer to this has been to shadow the trove, make those changes in the recipe, and rebuild. Of course, rebuilding all of apache to change a config file was a bit heavy handed, and rebuilding the kernel is even worse.
To address these needs (and it turns out, others as well), we added a new kind of package which inherits everything from another package, and then allows modifications to the inherited contents. When it's all done, a new package gets spit out with those changes.
This should be much easier to see with an example. I'm going to walk you through the steps of changing httpd.conf since that's such a common case.
First of all, we still need to shadow apache. I want to show the merge here, so I'm starting with an older version of httpd to let that step work.
bash# cvc shadow oot.rpath.org@oot:devtest httpd:source=conary.rpath.com@rpl:1/2.0.55-7 bash# cvc co httpd=oot.rpath.org@oot:devtest bash# cd httpdNow I'm going to erase the files we don't need anymore. We're going change httpd.conf and the recipe, so those can stay.
bash# cvc remove index.html Makefile.ssl modssl-req.conf httpd.init.patchGo ahead and edit httpd.conf. Make whatever changes you like; I'm just changing a comment at the top to prove I did something. The recipe needs to be modified as well since it's not building from source anymore. It's now a DerivedPackageRecipe which does nothing but install a new httpd.conf.
class Httpd(DerivedPackageRecipe): name = 'httpd' version = '2.0.55' def setup(r): r.addSource('httpd.conf', macros=True) r.Install('httpd.conf', '%(sysconfdir)s/httpd/conf/')
Most of the magic here is in the changed class of the package; making it a DerivedPackageRecipe tells conary to look for the latest httpd 2.0.55 on the parent branch of this shadow. The rest of this (all two lines) add httpd.conf as a source file and then installs it into the derived package.
Committing and cooking this as normal gives an httpd package with our new config file. If you're curious, here are some links:
- Source Package
- Modified recipe
- Built httpd trove
- Modified httpd.conf (look at the top of the file to see the change)
So, two lines to change the config files. A little bit easier than rebuilding the entire package from source. A nice feature of this is file versions make it easy to see what files have changed:
bash# conary rq httpd:runtime=oot.rpath.org@oot:devtest --file-versions --full-versions /etc/httpd/conf/httpd.conf /conary.rpath.com@rpl:devel//1//oot.rpath.org@oot:devtest/2.0.55-7.0.1-1 /etc/httpd/conf/magic /conary.rpath.com@rpl:devel/2.0.54-2-2
(Yeah, I snipped that output a bit) You can see from this that httpd.conf is a local modification while the magic file is inherited from upstream (unfortunately there is a bit of a problem in how file versions are calculated for files which multiple flavors on a shared path which makes this less impressive; Mihai is fixing!).
Merging to a new upstream version is pretty easy with this model. Since the recipe is completely new, there is no patch merging that needs to be done. Instead, conary just updates the version to reflect the new version we're deriving from.
bash# cvc merge bash# cvc diff (working version) Mon Jan 15 14:00:39 2007 (no log message) httpd.recipe: changed Index: httpd.recipe ==================================================================== contents(sha1) inode(mtime) --- httpd.recipe /conary.rpath.com@rpl:devel//1//oot.rpath.org@oot:devtest/2.0.55-7.0.1 +++ httpd.recipe @NEW@ @@ -1,6 +1,6 @@ class Httpd(DerivedPackageRecipe): name = 'httpd' - version = '2.0.55' + version = '2.0.59' def setup(r): r.addSource('httpd.conf', macros=True)
Commit, cook, and you have rPath's new httpd package with your configuration changes.
Right now the actions and policy objects which derived packages support are a bit limited. You can create new component (but not packages), move files between components, remove, install, and patch files in the build root, change file requirements, and a few other things. That list will certainly grow in the future though!