On properly packaging perl
| Did you know...? LWN.net is a subscriber-supported publication; we rely on subscribers to keep the entire operation going. Please help out by buying a subscription and keeping LWN on the net. |
The perl5-porters recently saw a rather acrimonious discussion on how the Red Hat and Fedora distributions choose to package the Perl language and associated modules. Things have calmed down (the parties have essentially agreed to disagree), but an interesting issue remains: what can development projects do if they're unhappy with how distributors are treating their code?
When Tom Christiansen gets irritated with somebody, one can generally be assured that they will know about it. In this case, Tom let the world know that he was not happy with the way Red Hat packages and distributes Perl. The complaint is that, if one installs "perl," one does not get the entire Perl 5.10 distribution. Tom says:
As it happens, it is possible to get the entire 5.10 distribution; one simply needs to install "perl-core" instead of plain "perl." Tom asserts that this arrangement is confusing; "perl-core" should not be a superset of "perl"; the package called simply "perl" (which is what most users will install) should be the thing that the Perl developers shipped. He says that Red Hat's arrangement causes confusion, with users not knowing if they actually have "Perl" or not.
Beyond that, it goes against the developers' intent, which included providing users with all the basic modules they needed from the outset. Many users, it seems, will not (or cannot) add extension modules to their systems; the Perl developers tried to ensure that these users would have a minimally-functioning system available to them. But, alas:
Tom "spot" Callaway explained Red Hat's packaging decisions:
Tom says that the package naming could be changed, but that would have a number of unfortunate effects on users. That said, there are some possibilities for improving the situation, but the best course is not entirely clear. Tom Christiansen was not completely happy with the explanation, but he also seemed to understand the pressures which led to Red Hat's way of doing things.
In the near term, it looks like things will not change a whole
lot be changing; see this comment for details. But
we're likely to see this kind of debate come back in the future. Distributors
serve as a sort of middleman, tweaking and refining upstream packages in
ways which they think improve things - either for their users or for themselves.
Distributor changes can include splitting up the package (as with Perl),
removing user-unfriendly messages (as has happened with cdrtools), making
the software more consistent with the rest of the distribution, fixing
security problems, removing software seen as legally problematic, and so
on. It's part of what distributors do, and users generally appreciate the
effort.
Upstream developers are harder to convince; they have released the software in the form that they think is best, so it can be discouraging to see others messing with it. Most developers suffer in stoic silence, taking comfort in the fact that their software is finding a wider audience. But others have taken various types of action in an attempt to influence how distributors treat their code.
Consider some examples. The kernel developers changed their release process dramatically with the (successful) goal of reducing the number of patches applied by distributors. Some developers do their own packaging. Jeff Waugh has suggested that much of the packaging role of distributions could be "disintermediated" entirely, with users routinely getting their software directly from its developers. Jörg Schilling inserted (widely ignored) statements into his code stating the alteration of some parts of the program would be a copyright violation. Firefox makes aggressive use of its trademarks to control the changes made by distributors. Daniel Bernstein took things to a (non-free) extreme, only allowing qmail to be distributed if it had not been modified at all; this restriction inhibited qmail development for years until the code was finally released into the public domain.
In the end, releasing code under a free license means giving up control
over what is done with it. So free software developers will always be at
the mercy of distributors, who will always have the right to make the
changes they think are necessary. The occasional grumble notwithstanding,
the system works pretty well; all of the parties involved share an interest
in having the software work as well as possible for their users.
(Log in to post comments)
On properly packaging perl
Posted Aug 21, 2009 21:15 UTC (Fri) by Baylink (guest, #755) [Link]
but TomC's right: they should have included CPAN itself in the base package, and perl-core is a bad choice of name, sounding like a *subset* of 'perl', not the superset it apparently is.
But then, Fedora's not a "real" distro, either, right?
On properly packaging perl
Posted Aug 21, 2009 21:50 UTC (Fri) by spot (guest, #15640) [Link]
On properly packaging perl
Posted Aug 21, 2009 21:55 UTC (Fri) by Baylink (guest, #755) [Link]
Did *that* change while I wasn't looking, too? Damn. :-)
On properly packaging perl
Posted Aug 21, 2009 22:06 UTC (Fri) by dowdle (subscriber, #659) [Link]
I don't often run Fedora on servers (or server virtual machines / containers) unless I have a need for something newer than RHEL5/CentOS5 and that isn't very often.
On properly packaging perl
Posted Aug 21, 2009 22:21 UTC (Fri) by quotemstr (subscriber, #45331) [Link]
I don't often run Fedora on servers (or server virtual machines / containers) unless I have a need for something newer than RHEL5/CentOS5 and that isn't very often.I regularly backport Fedora packages to RHEL when I need a newer version of a particular program or library. It works wonderfully (though of course, I have to manually monitor security lists for that package from that point onward.)
On properly packaging perl
Posted Aug 23, 2009 19:41 UTC (Sun) by mmahut (guest, #45550) [Link]
On properly packaging perl
Posted Aug 23, 2009 19:44 UTC (Sun) by quotemstr (subscriber, #45331) [Link]
On properly packaging perl
Posted Aug 24, 2009 14:18 UTC (Mon) by rahulsundaram (subscriber, #21946) [Link]
On properly packaging perl
Posted Aug 21, 2009 23:57 UTC (Fri) by Baylink (guest, #755) [Link]
On properly packaging perl
Posted Aug 21, 2009 22:06 UTC (Fri) by smoogen (subscriber, #97) [Link]
On properly packaging perl
Posted Aug 21, 2009 23:03 UTC (Fri) by BenHutchings (subscriber, #37955) [Link]
On properly packaging perl
Posted Aug 22, 2009 5:28 UTC (Sat) by drag (guest, #31333) [Link]
It's just something that won't ever be mainstream and is dubious for production environment.
On properly packaging perl
Posted Aug 22, 2009 8:03 UTC (Sat) by yoe (guest, #25743) [Link]
You are aware that Debian is used on several thousand systems in Munich too, right?
You are aware that the servers of Second Life all run Debian, right?
These are just the three first examples I could come up with off the top of my head. They all sound pretty 'mainstream' and 'production' to me.
I'm not saying Debian Rulez the World, but saying "won't ever be mainstream and is dubious for production" is just trolling.
On properly packaging perl
Posted Aug 22, 2009 8:13 UTC (Sat) by xanni (subscriber, #361) [Link]
On properly packaging perl
Posted Aug 29, 2009 18:55 UTC (Sat) by dirtyepic (guest, #30178) [Link]
On properly packaging perl
Posted Aug 22, 2009 8:15 UTC (Sat) by Sho (guest, #8956) [Link]
On properly packaging perl
Posted Aug 21, 2009 21:48 UTC (Fri) by spot (guest, #15640) [Link]
perl: depends on all subpackages provided in upstream tarball
perl-minimum (or perl-base, we're still discussing): minimum perl footprint for "base" functionality (e.g. no CPAN)
(To preempt the "why not in Fedora 12", we decided to hold off on making major perl changes during this short release cycle, since we have several big changes in addition to this to make, including the brown paper bag -DDEBUGGING fix)
Honestly, I would have preferred it if Tom Christiansen had filed a bug against our perl packages, to give us the opportunity to discuss this before jumping to huge conclusions and breathing fire, but since all of us are watching the p5p mailing list (if perhaps, not daily, at least weekly), we were able to respond to the issue in a reasonable amount of time.
Most of our followup discussion has happened on the fedora-perl-devel-list, you can see it here:
https://www.redhat.com/archives/fedora-perl-devel-list/20...
https://www.redhat.com/archives/fedora-perl-devel-list/20...
On properly packaging perl
Posted Aug 21, 2009 22:03 UTC (Fri) by paravoid (subscriber, #32869) [Link]
It may be better to lean on -base for consistency.
Perl Progress
Posted Aug 21, 2009 23:41 UTC (Fri) by corbet (editor, #1) [Link]
OK, so I missed the discussion on the Fedora-specific list. Guess I don't watch enough mailing lists...I stand corrected, sorry.
Perl Progress
Posted Aug 22, 2009 1:15 UTC (Sat) by mmcgrath (guest, #44906) [Link]
:: cough cough :: perhaps it's time to talk to people before writing about them and their intentions?
Perl Progress
Posted Aug 22, 2009 2:55 UTC (Sat) by sbergman27 (guest, #10767) [Link]
Perl Progress
Posted Aug 22, 2009 2:59 UTC (Sat) by mmcgrath (guest, #44906) [Link]
Yeah, that was constructive criticism. Had he done it (I suggested he do the same thing in one of his last columns), he wouldn't have had to retract one of his conclusions. I don't have time to donate so instead I donate my money.
RHEL6 based on Fedora 13?
Posted Aug 22, 2009 23:05 UTC (Sat) by gurulabs (subscriber, #10753) [Link]
RHEL 6 coming in May/June 2010?
RHEL 6?
Posted Aug 22, 2009 23:06 UTC (Sat) by gurulabs (subscriber, #10753) [Link]
RHEL 6 coming in May/June 2010?
RHEL 6?
Posted Aug 23, 2009 17:36 UTC (Sun) by spot (guest, #15640) [Link]
All I can say is that the Fedora perl maintainers contain the same folks who will maintain it for RHEL 6, and that this change will make it into RHEL 6.
On properly packaging perl
Posted Aug 21, 2009 23:40 UTC (Fri) by MattPerry (guest, #46341) [Link]
On properly packaging perl
Posted Aug 22, 2009 0:07 UTC (Sat) by khc (guest, #45209) [Link]
On properly packaging perl
Posted Aug 22, 2009 1:58 UTC (Sat) by MattPerry (guest, #46341) [Link]
On properly packaging perl
Posted Aug 22, 2009 6:18 UTC (Sat) by bojan (subscriber, #14302) [Link]
The upstream packages are raw materials. Distros peruse them to make a meaningful components for their system.
On properly packaging perl
Posted Aug 24, 2009 6:20 UTC (Mon) by nevyn (subscriber, #33129) [Link]
Except you might want to search for zero-install/autopackage/etc. where people have tried already ... and failed.
Or ask upstream developers why they don't ask for maintainership of their packages in (at least) Debian/Fedora[/EPEL].
On properly packaging perl
Posted Aug 29, 2009 19:05 UTC (Sat) by dirtyepic (guest, #30178) [Link]
On properly packaging perl
Posted Aug 22, 2009 1:32 UTC (Sat) by bojan (subscriber, #14302) [Link]
On properly packaging perl
Posted Aug 22, 2009 2:15 UTC (Sat) by MattPerry (guest, #46341) [Link]
Please elaborate.
On properly packaging perl
Posted Aug 22, 2009 6:15 UTC (Sat) by bojan (subscriber, #14302) [Link]
It also allows each distro to bring their own ideas forward. Some stick, some don't, but this is how they get proven (or not).
On properly packaging perl
Posted Aug 23, 2009 1:54 UTC (Sun) by drag (guest, #31333) [Link]
That does not mean that you have to do the packages separately for each and every distro. Also it does not mean that the majority of the packages should be customized to that degree.
There are a few things going on here...
Maybe quite a few application customizations that are compile-time options can be made to be runtime configuration options. That way you can do things upstream and still get the customization you need downstream. There probably has to be a understanding of what is a 'application' and what is a core system.
Even things that need to be compile time that does not mean that you still can't do packages upstream. Every decent package management system is perfectly capable of working on the source code level. Both Debs and RPMs have 'src' verions and tools.
The vast majority of upstream projects already package their software. Of course that package format is usually just tarballs. Even if binary compatibility is out of the question for some things it still does not mean packaging can't be done in a way that it is shared among all distros.
-------------------
As a end user one of the things that I always hate is having to give up Debian's packages. A few years ago trying to use CentOS or Fedora was hell for me because functionality and things I took for granted couldn't be done without huge amount of work.
Nowadays Fedora is closer to Debian, but that is taking a huge amount of effort and a huge amount of time on the part of the Fedora folks. It's years and it's still not quite up to the same level. I applaud the hard work and the dedication.. but essentially a lot of what they are doing is just doing work that was already done.
And it goes the other way also.
Debian could benefit heavily from the work Redhat and Fedora on putting into things like Virt-Manager and 389 Directory server. Sure Debian has Virt-Manager stuff packaged, but it's not really up to date and sucks compared to Fedora's version. There are a few differences that need to be taken into account... like Debian does not have the default firewall configuration that is used for Virt-Manager.. but that's not really that big of deal.
And 389 doesn't exist at all for Debian users as far as I can tell. If I was running a organization that already had LDAP and Kerberos intergration based around Debian systems (which is done in a very excellent way for Debian, btw. Does Fedora have things down to little details like Kerberos compatibility packages for OpenAFS so that K5-based users can still authenticaiton to use AFS?) and I want to take a look at 389 it would either mean I would be stuck building my own packages from scratch or switching out my operating system.
There is a huge amount of software that never really is that easily accessable to end users because it's never been packaged and tested very well. At work I am not in a position to spend time working out how to compile, package, and test applications... I have a busy enough time dealing with production issues and evaluating software that is already built and packaged by experts.
On properly packaging perl
Posted Aug 23, 2009 21:52 UTC (Sun) by bojan (subscriber, #14302) [Link]
Example: Recently we had a mass rebuild of packages for F-12, because the target will be i686, not i586 like in F-11. Now, if things were done the way you are suggesting, a decision like that would have to be coordinated among all distros, so that upstream starts shipping that at some magical point.
There is about zero chance of that happening.
Re: On properly packaging perl
Posted Aug 24, 2009 5:43 UTC (Mon) by nevyn (subscriber, #33129) [Link]
There probably has to be a understanding of what is a 'application' and what is a core system.
No, there doesn't. On a very superficial level, yeh, "everyone" knows glibc is part of the core system and gnucash is an application. But what about say python? And how do you split python, and why do you do those splits. This is all to do with the higher level applications and their deps. (much like the perl splits) ... which are completely different among distros. (or they'd be identical).
Realistically there's just not enough agreement to do "generic distro." packages, even if upstream was qualified/motivated to do them (which they aren't, or they'd be Fedora/Debian/etc. packagers already).
On properly packaging perl
Posted Aug 28, 2009 16:02 UTC (Fri) by stevem (subscriber, #1512) [Link]
On properly packaging perl
Posted Aug 22, 2009 0:13 UTC (Sat) by eparis123 (guest, #59739) [Link]
Really, no matter how smart of right someone is, the one who just destructively screams around loses.
Makes me remember the Reiser4 case.
Packaging and patching
Posted Aug 22, 2009 0:16 UTC (Sat) by rlk (guest, #47505) [Link]
A number of distributions patched Gutenprint 5.0.0 (which project I lead) to add margins for certain paper sizes (Gutenprint 5.0.0 with CUPS simply chopped off anything that wouldn't fit in the printable area of the page, and the way it was explained to me by one distributor was that their management got frustrated with printing letters that were chopped off) -- specifically, this patch forced margins for A4, letter, and legal size paper. This patch had the unfortunate side effect of making it impossible to print borderless to these page sizes even on printers that had no problem with it.
The problem is that they didn't consult with us first. We weren't aware of the level of impact this was having (arguably we should have, but it just happened that what I was printing wasn't causing me problems severe enough for me to care about). We only found out about it when people started complaining to us about not being able to print borderless; I had to look at the source RPM to figure out what was going on.
We wound up fixing this issue in a very different way in 5.0.1, and we release noted it extensively. Unfortunately, distributions never bothered removing the patch, so the broken behavior continued.
Finally, in 5.2, as part of a source code rearrangement, we were able to do something about it: we moved the file to a different location in the source tree, and I inserted some comments into the offending parts of the XML source that guaranteed that any attempt to apply the patch would break, forcing someone to take a look at it and (hopefully) reading our instructions that this was a really bad thing to do, and in any event, the problem underlying this was solved.
We've always had a section in our README specifically targeted to packagers and tried to make it very visible at the top of the file that they should read that section. Among other things, we request that distributors join our development mailing list and discuss changes with us. In this case, it would have raised the urgency of a real fix, and even if distributors decided to temporarily go with the fix, we would have known what was going on, would have known who to redirect bugs to, and would have told them very clearly that in 5.0.1 that patch should go away. I tried beefing it up. I don't know what more we can do. When users complain about problems on prepackaged bits that look fishy to me, I often do tell them to install from source to see if the problem goes away -- not too different in principle from what Joerg Schilling did -- because I don't want to try to debug someone else's patches. I don't agree with everything that he's done, but I certainly sympathize with the kind of frustration this can engender.
The reality is that under the GPL (and most other FOSS licenses), there is nothing legally we could do (I believe -- IANAL) to prevent this, and people do have a right to modify the code. However, from a *pragmatic* point of view, distributors should work with upstream developers and try to understand why developers make the decisions we do. That would also help developers understand the issues distributors face.
Re: Packaging and patching
Posted Aug 24, 2009 6:03 UTC (Mon) by nevyn (subscriber, #33129) [Link]
It's frustrating when distributors change things without discussion with upstream developers, even when it's a bugfix.
Packagers don't live in an ivory tower, and the vast majority will happily accept at least co-maintainers from upstream (if not just give them the package).
We wound up fixing this issue in a very different way in 5.0.1, and we release noted it extensively. Unfortunately, distributions never bothered removing the patch, so the broken behavior continued.
So you go to all the effort of finding the bug, and documenting the correct fix etc. ... but don't bother to spend 5 minutes opening a distro. bug saying "X is the wrong fix, this is the upstream fix".
Re: Packaging and patching
Posted Aug 24, 2009 10:56 UTC (Mon) by rlk (guest, #47505) [Link]
Re: Packaging and patching
Posted Aug 24, 2009 15:26 UTC (Mon) by mcopple (guest, #2920) [Link]
So you go to all the effort of finding the bug, and documenting the correct fix etc. ... but don't bother to spend 5 minutes opening a distro. bug saying "X is the wrong fix, this is the upstream fix".
-------
It seems to me that the correct workflow would be for downstream packagers / maintainers to discipline themselves to check for upstream changes on a regular basis, rather than relying on upstream to figure out who is packaging their product and notify.
It is incumbent on the upstream developer to have channels available to get the word out -- a developer's list, a bug tracking system, release notes, etc. But it is not reasonable to expect upstream to check the bug trackers of all of the scores of active distributors out there.
Re: Packaging and patching
Posted Aug 29, 2009 19:30 UTC (Sat) by dirtyepic (guest, #30178) [Link]
At the very least they should find the time to browse the release notes.
Re: Packaging and patching
Posted Aug 24, 2009 17:20 UTC (Mon) by jimi (guest, #6655) [Link]
So you go to all the effort of finding the bug, and documenting the correct fix etc. ... but don't bother to spend 5 minutes opening a distro. bug saying "X is the wrong fix, this is the upstream fix".
This is absolutely the wrong attitude and workflow pattern. The software developers should have no responsibility at all to file bugs to various distributors' mailing lists. The job of a distributor is to package the original source and optionally patch *and notify the upstream developers* of any bugs. This notification must go from distributor to developer, not the other way around. The article notes that Red Hat would have preferred that the perl guys had filed a bug in the Red Hat bug tracking system. That is completely the wrong expectation. It is Red Hat who should possibly be filing bugs, not the other way around.
There is a possible exception to this rule: when the upstream developers are unreachable. In that case the distributor may choose to become the new maintainers of the software.
Re: Packaging and patching
Posted Aug 24, 2009 17:52 UTC (Mon) by dlang (guest, #313) [Link]
Re: Packaging and patching
Posted Aug 24, 2009 18:41 UTC (Mon) by nevyn (subscriber, #33129) [Link]
So you go to all the effort of finding the bug, and documenting the correct fix etc. ... but don't bother to spend 5 minutes opening a distro. bug saying "X is the wrong fix, this is the upstream fix".This is absolutely the wrong attitude and workflow pattern. The software developers should have no responsibility at all to file bugs to various distributors' mailing lists. The job of a distributor is to package the original source and optionally patch *and notify the upstream developers* of any bugs. This notification must go from distributor to developer, not the other way around.
FFS, read what I wrote. Upstream had already seen the patch the distro. did ... and added release notes about how they'd fixed it differently (to stop other problems). But couldn't spend the couple of minutes directly telling the distro? ... and this is the distros. fault for "not doing it's job". I think not.
But in the general case ... Yes, in a software developers perfect world they'd release upstream code full of "features". Distros. would then fix everything for them, and spend huge amounts of time rewritting those patches to make the developers happy while they spend their time adding more features.
Back in the real world packagers are often only spending a fraction of their time doing packaging, often among a bunch of packages, so that "blah install random-crack" works. They aren't unpaid upstream minions, and they sure as hell aren't paid for their "job". Yes, they are hopefully following development in some way, but they are still unpaid humans and can easily miss things.
On the other side upstream developers are often spending a majority of their time on a 1-3 projects. So those upstream developers spending an hour a month or so to make sure the packaging in (at least) the major distros. isn't broken in some way, doesn't seem like too much to ask IMNSO.
And, yeh, for larger projects (like perl) you might expect that the distros. have more resources to put towards upstream ... but at the same time upstream will often have lots more resources to look downstream.
Re: Packaging and patching
Posted Aug 25, 2009 0:10 UTC (Tue) by rlk (guest, #47505) [Link]
Checking all of the major distributions -- let's say, Fedora, Ubuntu, OpenSUSE, Debian unstable, Mandriva -- is a lot of work. It also means having the disks to do a lot of installations, the time to spend downloading the latest bits, the time to run tests, and so forth. No, this isn't practical.
Our automated test suite wouldn't have caught this in any case, since the problem was that the definition of letter and A4 paper was changed in a way that's legal within Gutenprint (and would therefore not fail functional and regression testing) but which would yield unexpected results.
The interaction I would have preferred is:
* Distributor files a bug/complains on the -devel mailing list that the behavior is unacceptable.
* We reply that it's behaving as designed.
* Distributor says "well, maybe so, but but it's still broken and we need to do something about it."
* Maybe we figure out the ultimate solution and get a fix in time for the next release (in which case everyone's happy) or maybe not, and distributor proposes the particular hack they actually did. More likely the latter, since the ultimate fix needed quite a bit of testing to get right.
* Distributor probably decides, after discussion, to accept that borderless printing won't work to those paper sizes and tells us that they're going to release with it. We suggest that they release note what's going on and keep checking back for a real fix.
* We get the fix out a bit quicker than we ultimately did, since we realize how critical it is in the real world; distributor picks it up and either patches the current release or at least makes sure the next release has the real fix (without applying the bogus patch again).
Re: Packaging and patching
Posted Aug 27, 2009 7:42 UTC (Thu) by job (guest, #670) [Link]
Otherwise all security that surround distributions is for nothing if they package whatever comes upstream (via unsecured FTP sites and whatever).
Not even reading the release notes is in my opinion very far from having a clue what changes are applied!
Packaging and patching
Posted Aug 24, 2009 6:17 UTC (Mon) by dgm (subscriber, #49227) [Link]
In my opinion, a packager should NEVER, EVER patch something. If it doesn't work, complain to upstream until it's fixed. If it's not fixed, drop the package or revert to a previous version. The case of security fixes is not different.
But that's just my opinion, of course.
Packaging and patching
Posted Aug 24, 2009 8:44 UTC (Mon) by Trou.fr (subscriber, #26289) [Link]
And regarding security issues, i'd rather have a quick patch by the packager than a week late patch by upstream.
Packaging and patching
Posted Aug 24, 2009 12:19 UTC (Mon) by dgm (subscriber, #49227) [Link]
A quick fix made by a packager that doesn't understand the implications can create more problems that it solves (Debian OpenSSL anyone?) Also your arguments is based on the *false* assumption that packagers do release fixes faster than upstream.
And now that we're at it, instead of just packaging everything under the sun, why not just talk to upstream regarding their stance behind the code _before_ creating a package for it?
Packaging and patching
Posted Aug 24, 2009 14:53 UTC (Mon) by foom (subscriber, #14868) [Link]
Why? Because I like distros that have everything under the sun, and work as an intermediary between possibly-uncooperative upstreams and me. And who do the work that most upstreams don't do to make the software work together easily and consistently.There have been a couple disastrous changes, indeed, but the vast majority of minor modifications I greatly appreciate having.
Packaging and patching
Posted Aug 25, 2009 7:04 UTC (Tue) by dgm (subscriber, #49227) [Link]
I tend to think that the 80/20 rule also applies to software (yes, all software), thus much of what is there is crap. A process that sinks crap and lets the good stuff stay afloat is really needed to ensure a sane evolution of the good software. Some distributions (Debian and Ubuntu at least) let this up to the user, in the form of a popularity contest. Wrong. This is _the_ task that the packager should be doing.
All those "minor modifications" just make things _unnecessarily_ harder for everybody.
Packaging and patching
Posted Aug 25, 2009 15:38 UTC (Tue) by foom (subscriber, #14868) [Link]
Well, I like my software modified and configured to work better as a whole. And I like that no matter what software I want (even if you consider it crap), it's almost certainly already available in Debian.There's space in the world for multiple distributions with different goals, and I bet there's at least one which doesn't modify upstream source code that you can use.
Packaging and patching
Posted Aug 27, 2009 23:16 UTC (Thu) by branden (guest, #7029) [Link]
As was widely reported at the time, Debian's package maintainer *did* take
the patch to the documented upstream development list. Communications
took place, but went awry, with each side not completely understanding the
other. Ben Laurie later waded in to cluck that distributors suck, and
that the patch should have been sent to the list that all the cool OpenSSL
upstream kids read. (The existence of said list was a well-kept secret
from the general public until the bug in question blew up in Debian's
face.)
There was lots of fail in the Debian OpenSSL situation, but failure to run
a patch by the upstream developers was not a component of it.
Packaging and patching
Posted Aug 28, 2009 0:12 UTC (Fri) by rlk (guest, #47505) [Link]
In the Gutenprint example, nobody ever attempted to contact us (we have a development mailing list that's noted on our web site that's open subscription), even to notify us that they were making the change in question. We were left to find out about it from complaints from unhappy users.
Packaging and patching
Posted Aug 29, 2009 20:51 UTC (Sat) by dirtyepic (guest, #30178) [Link]
I can patch a freetype bug in Gentoo and have it out to users sometime within the next hour (whenever the next rsync mirror update is). I don't think they can make a release that fast. In fact, we patched a security vulnerability in freetype-2.3.9 back in May, and as of now there still hasn't been a new upstream release. So, should we drop freetype or fork it? (note: i don't mean to pick on freetype here, it's just a package i happen to maintain)
And by "release" I mean get the fix out to the people encountering the bug, not fix it in the repo for whenever the next release may be. If you have some other definition then please share it.
We also recently patched fontconfig because recent upstream changes were causing problems with our sandboxed build environment (packages calling fc-cache during `make install` would fail because fc-cache now runs chmod on /var/cache/fontconfig and we don't allow packages to change permissions on files or directories they don't own outside of the DESTDIR during install). I did let upstream know about the issue, but ultimately I fixed it locally because it was our policy causing the issue, not upstream. Debian also modifies most packages to comply with their policies. Expecting upstream to accommodate the (sometimes conflicting) packaging policies of every major distro is ridiculous.
Trust me, we would love it if everything worked the way we wanted out of the box. We don't go through the trouble of maintaining large patchsets for the sheer enjoyment it gives us.
Packaging and patching
Posted Aug 24, 2009 15:32 UTC (Mon) by mcopple (guest, #2920) [Link]
When downstream discovers the problem, open a bug for it with upstream. If upstream doesn't have it fixed by the time the distributor needs it, fix it yourself, display it prominently in the release notes, notify upstream you've done it.
If upstream still doesn't do anything about the problem, then it is upstream's problem; you keep the patch in place and your customers are satisfied. If upstream resolves the issue to the distributor's satisfaction, use upstream and quietly drop the downstream patch.
Packaging and patching
Posted Aug 24, 2009 18:46 UTC (Mon) by nevyn (subscriber, #33129) [Link]
Sure, Fedora should just fork python or drop it completely (upstream have refused to take multilib seriously). Not to mention what awesome support you'd get off all upstreams for everything in RHEL-5 atm.
You seem to have awesome ideas, do you produce a newsletter I can subscribe to?
On properly packaging perl
Posted Aug 22, 2009 0:59 UTC (Sat) by dwheeler (guest, #1216) [Link]
Although it wouldn't have helped this case, one thing that would help would be if FLOSS packages followed the usual conventions for source releases. Stuff like, support ./configure --prefix=/usr ; make ; make install, and support the DESTDIR convention (which both RPM and deb essentially require). So many source packages fail to support the DESTDIR convention that I ended up having to create Auto-DESTDIR, a program to automatically add DESTDIR support to programs that don't support it natively.
When creating the package takes a thousand lines, it's easy to justify five more, and it's easy to miss a "bad" change later. In contrast, when the packaging work is only a few lines long, local patches are a lot more glaring - and in my experience, less likely.
On properly packaging perl
Posted Aug 22, 2009 6:24 UTC (Sat) by pabs (subscriber, #43278) [Link]
On properly packaging perl
Posted Aug 22, 2009 22:30 UTC (Sat) by RobWilco (guest, #40828) [Link]
On a related thought, I see no good page for helping python projects to be friendly toward distributions. I can see the python buildout system getting momentum among developers but no standard way to transform a buildout into a debian package.
On properly packaging python
Posted Aug 23, 2009 5:39 UTC (Sun) by zooko (guest, #2589) [Link]
On properly packaging perl
Posted Aug 23, 2009 8:28 UTC (Sun) by Cato (guest, #7643) [Link]
Auto-DESTDIR
Posted Aug 23, 2009 12:18 UTC (Sun) by dwheeler (guest, #1216) [Link]
I think so, too. I've sent some emails to people who people I thought would be interested in integrating it into their systems. I've started the process of trying to get auto-destdir packaged into Fedora; that doesn't integrate it into checkinstall, but it makes it more accessible to a few. If others would like to package it for other distros, or help patch it into other programs, that'd be great.
On properly packaging perl - Auto-DESTDIR now in Fedora repository
Posted Sep 14, 2009 15:28 UTC (Mon) by dwheeler (guest, #1216) [Link]
Auto-DESTDIR is now in the standard (yum) repository of Fedora 10, Fedora 11, and the upcoming Fedora 12 (aka rawhide). Barring catastrophe, it'll even be on the Fedora 12 DVDs/CDs. I intend to create EPEL packages, too, so Red Hat Enterprise Linux and CentOS users can easily use them too. I'd love to see packaging for Debian, Ubuntu, SuSE, etc - volunteers welcome! Hopefully this will help package and install programs.
On properly packaging perl
Posted Aug 27, 2009 23:27 UTC (Thu) by branden (guest, #7029) [Link]
with a slab of advice counseling original developers to avoid cultivating
ignorance and idiosyncrasy.
An essential aspect of software engineering is learning how to retain
one's healthy ego without behaving like an arrogant cowboy. In my limited
experience, people working for distributions learn that faster, because
it's easier to learn humility when you're beholden to three groups of
people at once: your upstream, your peers, and your users. Too often,
especially with one-man software projects, a developer simply does not
grow beyond the Raymondesque "scratching one's own itch" phase of
development.
Some upstream developers actively monitor the bug-tracking lists of major
distributors relevant to their code. (I'd like to single out Thomas
Dickey for praise in this department--he's a one-man operation who seems
to do everything right.) Again, in my experience, this tightens the bond
between distributor and upstream and makes for a happier experience. When
both sides lift a finger, it's easier for them to shake hands.
On properly packaging perl
Posted Aug 24, 2009 13:26 UTC (Mon) by zsouthboy (guest, #60295) [Link]
I don't know CPAN, etc., but this sounds like the equivalent of leaving out a standard part of the language, right? Like packaging python (which I am familiar with) without, say, the Math library?
How can you expect your perl script to run anywhere if the default libraries can't be depended on?
On properly packaging perl
Posted Aug 24, 2009 14:38 UTC (Mon) by djpig (guest, #18768) [Link]
I'm not familiar with perl - can someone explain why they would bother stripping modules out of the standard distribution of perl?Here on my Debian lenny system perl + perl-modules + perl-base currently use about 36MB, of which perl-base uses 4.6 MB. And perl-base gives you a fully functional perl interpreter... Sure, on a modern server or desktop 36MB is not a lot, but Debian tries to support as many use cases as possible, and so it makes sense to define a basic set of perl modules that are always available, so people can use them in e.g. package scripts without needing to depend on the full Perl. (The split in perl and perl-modules I can't explain, but they depend on each other so you can't install one without the other anyway)
On properly packaging perl
Posted Aug 27, 2009 2:50 UTC (Thu) by BenHutchings (subscriber, #37955) [Link]
On properly packaging perl
Posted Aug 24, 2009 15:22 UTC (Mon) by bferrell (subscriber, #624) [Link]
On properly packaging perl
Posted Aug 28, 2009 19:39 UTC (Fri) by oak (guest, #2786) [Link]
> out a standard part of the language, right? Like packaging python
> (which I am familiar with) without, say, the Math library?
Python's actually similarly split in Debian based distros.
In Ubuntu python-minimal is essential[1]:
http://packages.ubuntu.com/karmic/python-minimal
http://packages.ubuntu.com/karmic/python
[1] like perl-base is on all Debian derivatives. I just wonder why one is
*-base and another *-minimal...
Mixing CPAN and packaging
Posted Aug 24, 2009 15:22 UTC (Mon) by zlynx (guest, #2285) [Link]
I cannot begin to tell you how many problems CPAN has caused when mixed with RPM packages coming from the distro and/or from alternate repositories.
CPAN wants to upgrade packages it has no business touching. It wants to install things into locations that disagree with RPM. It leads to multiple copies of the same thing installed into slightly different places making it impossible to tell exactly what copy of SSL is crashing your app.
Use RPM *or* CPAN but not both.
Mixing CPAN and packaging
Posted Aug 24, 2009 22:33 UTC (Mon) by Cato (guest, #7643) [Link]
CPAN is required for all third party software
Posted Aug 27, 2009 7:52 UTC (Thu) by job (guest, #670) [Link]
Every program written in Perl that you would want to install manually comes with install instructions that includes using CPAN to get the required libraries installed. That's the culture and it has served well for the past fifteen years or so. The alternative would be to include all dependencies in all programs, which would be worse.
A system without CPAN would leave many beginning system administrators very confused on how to proceed and give Perl programs an unfounded reputation of being hard to install. There should at least be a stub installed that brings in the real stuff when called for or at least explains the situation.
CPAN is required for all third party software
Posted Aug 27, 2009 8:21 UTC (Thu) by Cato (guest, #7643) [Link]
What would be even nicer is a simple tool that, given a CPAN package name, first tries the distro's packaged versions of CPAN packages, and only if they don't exist tries CPAN directly.
CPAN is required for all third party software
Posted Aug 28, 2009 15:41 UTC (Fri) by epa (subscriber, #39769) [Link]
get the package from CPAN and build a package of it using a tool like
cpanflute2, cpan2rpm, makerpm.pl, etc etc.
On properly packaging perl
Posted Sep 10, 2009 20:35 UTC (Thu) by philips (guest, #937) [Link]
You are right in general.
But you are wrong in the context. We are not talking about a package and random modification to it. We are talking about a programming language.
Perl 5 doesn't have any specification. There are not distro types. Perl 5 is a Perl 5 only when the whole shebangs from tarball is installed. Otherwise you run into the possibility that 3rd party script depending on standard (present in tarball) module will not run - because RH decided that it's not needed.
To exemplify. Imagine that RH had installed GCC but decided to exclude /usr/include/stdio.h. Yes, they have very right to do it. It's all open source. But it would break compilation of more or less every C/C++ program.
Perl 6 which has proper specification (separating mandatory modules from extras) shouldn't have the problem. But for Perl 5 - the tarball is the specification. Deviating from it means only pains for its users.