Discussion:
Bundled Provides Libraries and Versioning
(too old to reply)
Adam Miller
2017-07-07 16:55:50 UTC
Permalink
Hello all,
In today's FESCo meeting we discussed the fact that there are many
RPMs currently in Fedora (a reported 244 in Rawhide currently) that
are defining a `Provides: bundled(<lib>) = <version>` but excluding
the version completely[0][1]. This removes that ability to properly
perform source code auditing and security vulnerability tracking.

My question to the Fedora Contributor Community is, how should we
handle this? Is this something that should just simply be fixed by the
packages currently violating the Guidelines, should the Guidelines be
altered in a way that makes this easier to deal with for Packagers but
also provides what is needed for auditing and vulnerability tracking,
or is there simply clarification needed by what is required in the
<version> field?

I look forward to the discussion.

Thank you,
-AdamM


[0] - https://pagure.io/fesco/issue/1734
[1] - https://pagure.io/packaging-committee/issue/696
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an em
Jerry James
2017-07-07 17:05:37 UTC
Permalink
On Fri, Jul 7, 2017 at 10:55 AM, Adam Miller
Post by Adam Miller
Hello all,
In today's FESCo meeting we discussed the fact that there are many
RPMs currently in Fedora (a reported 244 in Rawhide currently) that
are defining a `Provides: bundled(<lib>) = <version>` but excluding
the version completely[0][1]. This removes that ability to properly
perform source code auditing and security vulnerability tracking.
My question to the Fedora Contributor Community is, how should we
handle this? Is this something that should just simply be fixed by the
packages currently violating the Guidelines, should the Guidelines be
altered in a way that makes this easier to deal with for Packagers but
also provides what is needed for auditing and vulnerability tracking,
or is there simply clarification needed by what is required in the
<version> field?
How many of those are bundled(jquery)? A couple of years ago, those
of us with packages that generate documentation via doxygen were asked
to add that to our spec files. (I no longer remember who asked us,
sorry.) The reasoning was that someday, somebody would want to do
something about fixing doxygen to point to a system version of jquery,
and adding those Provides would make the process of finding the
affected packages easier. I haven't heard anything more since then,
so I don't know if anybody is even working on the issue.
--
Jerry James
http://www.jamezone.org/
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fe
Jerry James
2017-07-07 17:10:46 UTC
Permalink
On Fri, Jul 7, 2017 at 10:55 AM, Adam Miller
Post by Adam Miller
Hello all,
After which my response was met with:

------------------------------------------------------------------------------
Your message to the packaging mailing-list was rejected for the following
reasons:

The message is not from a list member

The original message as received by Mailman is attached.
------------------------------------------------------------------------------

You cross-posted to closed lists. Please don't do that unless you
know for certain that the lists have identical membership, which
pretty much obviates the need for posting to multiple lists. If you
want to send the same message to multiple closed mailing lists, then
send copies of it, one at a time, to each list. Thank you,
--
Jerry James
http://www.jamezone.org/
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedorap
Adam Miller
2017-07-07 21:35:11 UTC
Permalink
Post by Jerry James
On Fri, Jul 7, 2017 at 10:55 AM, Adam Miller
Post by Adam Miller
Hello all,
------------------------------------------------------------------------------
Your message to the packaging mailing-list was rejected for the following
The message is not from a list member
The original message as received by Mailman is attached.
------------------------------------------------------------------------------
You cross-posted to closed lists. Please don't do that unless you
know for certain that the lists have identical membership, which
pretty much obviates the need for posting to multiple lists. If you
want to send the same message to multiple closed mailing lists, then
send copies of it, one at a time, to each list. Thank you,
Apologies, it was requested in the FESCo meeting that I CC the FPC so
I did. I wasn't aware of the restrictions on the mailing list. I'll be
sure to verify in the future.

-AdamM
Post by Jerry James
--
Jerry James
http://www.jamezone.org/
_______________________________________________
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to d
Randy Barlow
2017-07-07 17:35:36 UTC
Permalink
Post by Adam Miller
My question to the Fedora Contributor Community is, how should we
handle this?
I am guilty of doing this a few times in a couple of my packages. My
reasons when I did it were that I didn't know the version, and didn't
know an easy way to determine the version. I figured it was better to
at least declare that I'm bundling it with uncertainty on the version
than to not declare at all.
Michael Catanzaro
2017-07-07 19:31:24 UTC
Permalink
On Fri, Jul 7, 2017 at 12:35 PM, Randy Barlow
Post by Randy Barlow
I am guilty of doing this a few times in a couple of my packages. My
reasons when I did it were that I didn't know the version, and didn't
know an easy way to determine the version. I figured it was better to
at least declare that I'm bundling it with uncertainty on the version
than to not declare at all.
We do this for WebKit, which bundles ANGLE and various other crap
Google libraries, because crap Google libraries do not have releases or
versions. They do generally have tags corresponding to Chrome releases,
but I think we generally use git snapshots. So just thinking up a
version number to use for the provides is not easy.

Another problem is selecting the name for the provides. E.g. pick
between bundled(ANGLE) or bundled(libangle). With the next evince
update, evince will bundle a LZMA decoder. Should it be bundled(LZMA)
or bundled(lzma-sdk) or bundled(7zip)? Some of these options seem
better than others, but all are reasonable and maintainers are likely
to choose different ones....

Michael
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe
Björn Persson
2017-07-08 12:39:03 UTC
Permalink
Post by Michael Catanzaro
Another problem is selecting the name for the provides. E.g. pick
between bundled(ANGLE) or bundled(libangle). With the next evince
update, evince will bundle a LZMA decoder. Should it be bundled(LZMA)
or bundled(lzma-sdk) or bundled(7zip)? Some of these options seem
better than others, but all are reasonable and maintainers are likely
to choose different ones....
Indeed, that's the big flaw with this way of tracking bundled stuff,
which I pointed out when it was first proposed. It seems that nobody has
a good answer.

Björn Persson
Jason L Tibbitts III
2017-07-07 17:43:48 UTC
Permalink
[...]
AM> RPMs currently in Fedora (a reported 244 in Rawhide currently) that
AM> are defining a `Provides: bundled(<lib>) = <version>` but excluding
AM> the version completely[0][1]. This removes that ability to properly
AM> perform source code auditing and security vulnerability tracking.

I would argue that it doesn't remove the ability, but that it does make
it more difficult to do in an automated fashion. Basically you can see
that something has a bundled library but then you need to do manual
inspection to go further.

AM> My question to the Fedora Contributor Community is, how should we
AM> handle this?

Identify and mail lists of the problematic packages to devel (using
find-package-maintainers from
https://pagure.io/fedora-misc-package-utilities if possible). Figure
out if there are any cases which aren't easy to fix for some reason.

If there are any, then see if a change is needed to accommodate.

If I had to hazard a guess, I would say that there are at least some
cases where it's not really obvious what version to use. This would
make sense in the case of a fork that's undergone significant rewriting.
Though I wonder if any bundled(X) tag is even warranted in that case.

Alternatively, say that you don't have to specify a version, but if you
don't then you will get every related security bug filed against your
package instead of having those filtered by version.

- J<
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@li
Björn Persson
2017-07-08 12:13:59 UTC
Permalink
Post by Jason L Tibbitts III
Alternatively, say that you don't have to specify a version, but if
you don't then you will get every related security bug filed against
your package instead of having those filtered by version.
Perhaps with a notice included in each such bug report, along the lines
of "Because the version of the bundled library is unspecified, we must
assume that it is a vulnerable version.", to make people aware that
they can avoid irrelevant bug reports by adding a version number if one
exists.

Björn Persson
Ville Skyttä
2017-07-08 15:24:52 UTC
Permalink
7.7.2017 20.45 "Jason L Tibbitts III" <***@math.uh.edu> kirjoitti:


I would argue that it doesn't remove the ability, but that it does make
it more difficult to do in an automated fashion. Basically you can see
that something has a bundled library but then you need to do manual
inspection to go further.


I think the versioning isn't worth much at all.

If the bundled version corresponds to an upstream release to an extent that
it can be called that version, and checks like the discussed one could be
skipped just by looking at the version label, then it must be practically
the same. So why is it bundled in the first place?

On the other hand if there is a "good" reason it is bundled, that reason
quite probably is that it is a modified version. So it's different than the
upstream one, and thus knowledge whether an upstream release is vulnerable
or not cannot be just assumed based on the version label a packager has
attached to it. It needs to be checked anyway.
Dominik 'Rathann' Mierzejewski
2017-07-10 11:18:36 UTC
Permalink
On Saturday, 08 July 2017 at 17:24, Ville Skyttä wrote:
[...]
Post by Ville Skyttä
If the bundled version corresponds to an upstream release to an extent that
it can be called that version, and checks like the discussed one could be
skipped just by looking at the version label, then it must be practically
the same. So why is it bundled in the first place?
There can be many reasons: copylibs, no upstream support for building as
shared library, no stable API/ABI, etc. Many of them are listed on the
wiki page: https://fedoraproject.org/wiki/Bundled_Libraries_Virtual_Provides

Regards,
Dominik
--
Fedora http://fedoraproject.org/wiki/User:Rathann
RPMFusion http://rpmfusion.org
"Faith manages."
-- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to de
Bastien Nocera
2017-07-10 11:56:52 UTC
Permalink
----- Original Message -----
Post by Dominik 'Rathann' Mierzejewski
[...]
Post by Ville Skyttä
If the bundled version corresponds to an upstream release to an extent that
it can be called that version, and checks like the discussed one could be
skipped just by looking at the version label, then it must be practically
the same. So why is it bundled in the first place?
There can be many reasons: copylibs, no upstream support for building as
shared library, no stable API/ABI, etc. Many of them are listed on the
wiki page: https://fedoraproject.org/wiki/Bundled_Libraries_Virtual_Provides
There's no mention of zlib or LZMA SDK libs in there. Should those be added?
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-leave@
Dominik 'Rathann' Mierzejewski
2017-07-11 18:31:05 UTC
Permalink
Post by Bastien Nocera
----- Original Message -----
[...]
Post by Bastien Nocera
Post by Dominik 'Rathann' Mierzejewski
There can be many reasons: copylibs, no upstream support for building as
shared library, no stable API/ABI, etc. Many of them are listed on the
wiki page: https://fedoraproject.org/wiki/Bundled_Libraries_Virtual_Provides
There's no mention of zlib or LZMA SDK libs in there. Should those be added?
Yes, please do.

Regards,
Dominik
--
Fedora http://fedoraproject.org/wiki/User:Rathann
RPMFusion http://rpmfusion.org
"Faith manages."
-- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject

Adam Miller
2017-07-10 16:40:12 UTC
Permalink
Post by Jason L Tibbitts III
I would argue that it doesn't remove the ability, but that it does make
it more difficult to do in an automated fashion. Basically you can see
that something has a bundled library but then you need to do manual
inspection to go further.
I think the versioning isn't worth much at all.
If the bundled version corresponds to an upstream release to an extent that
it can be called that version, and checks like the discussed one could be
skipped just by looking at the version label, then it must be practically
the same. So why is it bundled in the first place?
On the other hand if there is a "good" reason it is bundled, that reason
quite probably is that it is a modified version. So it's different than the
upstream one, and thus knowledge whether an upstream release is vulnerable
or not cannot be just assumed based on the version label a packager has
attached to it. It needs to be checked anyway.
The main motivation for bundling as of late is golang[0], it's
extremely common out in the community for software to pull in
"Vendored" libraries even if they are exact copies of remote upstreams
(this is common with tools like godep[1]) because golang is statically
compiled by default (you can dynamically link with gcc-go and I
*think* recent releases of cgo but I've yet to find a golang project
that officially uses or endorses it's use). It's also extremely
painful to unbundle these as some more popular software written in
golang such as docker[2], kubernetes[3], and openshift[4] have upwards
of 100 bundled libs (over 1000 for OpenShift).

-AdamM

[0] - https://golang.org/
[1] - https://github.com/tools/godep
[2] - http://pkgs.fedoraproject.org/cgit/rpms/docker.git/tree/docker.spec
[3] - http://pkgs.fedoraproject.org/cgit/rpms/kubernetes.git/tree/kubernetes.spec
[4] - http://pkgs.fedoraproject.org/cgit/rpms/origin.git/tree/origin.spec
Post by Jason L Tibbitts III
_______________________________________________
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe se
Randy Barlow
2017-07-10 22:18:17 UTC
Permalink
Post by Adam Miller
The main motivation for bundling as of late is golang[0], it's
extremely common out in the community for software to pull in
"Vendored" libraries even if they are exact copies of remote
upstreams
(this is common with tools like godep[1]) because golang is
statically
compiled by default (you can dynamically link with gcc-go and I
*think* recent releases of cgo but I've yet to find a golang project
that officially uses or endorses it's use). It's also extremely
painful to unbundle these as some more popular software written in
golang such as docker[2], kubernetes[3], and openshift[4] have
upwards
of 100 bundled libs (over 1000 for OpenShift).
On top of these problems I have also observed a trend away from having
releases and versions at all. Lots of golang programs just depend on
git commit hashes of libraries that don't make releases. I've also
observed this problem in some JavaScript libraries.
Jason L Tibbitts III
2017-07-10 22:28:37 UTC
Permalink
RB> On top of these problems I have also observed a trend away from
RB> having releases and versions at all. Lots of golang programs just
RB> depend on git commit hashes of libraries that don't make
RB> releases. I've also observed this problem in some JavaScript
RB> libraries.

That's true, but we have a scheme for versioning packages containing
such things, and I see no reason that scheme isn't applicable anywhere
we might need to make reference to a version of a particular piece of
software.

- J<

RB> _______________________________________________ devel mailing list
RB> -- ***@lists.fedoraproject.org To unsubscribe send an email to
RB> devel-***@lists.fedoraproject.org


--
Jason L Tibbitts III - ***@math.uh.edu - 713/743-3486 - 660PGH
System Manager: University of Houston Department of Mathematics
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Kevin Kofler
2017-07-09 22:36:52 UTC
Permalink
Post by Adam Miller
In today's FESCo meeting we discussed the fact that there are many
RPMs currently in Fedora (a reported 244 in Rawhide currently) that
are defining a `Provides: bundled(<lib>) = <version>` but excluding
the version completely[0][1]. This removes that ability to properly
perform source code auditing and security vulnerability tracking.
My question to the Fedora Contributor Community is, how should we
handle this? Is this something that should just simply be fixed by the
packages currently violating the Guidelines, should the Guidelines be
altered in a way that makes this easier to deal with for Packagers but
also provides what is needed for auditing and vulnerability tracking,
or is there simply clarification needed by what is required in the
<version> field?
A version number may not even exist at all. Not all code that people copy is
a library with a version number. Copylibs often don't bother doing releases
because everyone just embeds it as a git submodule or checks out some random
revision to copy into their own SCM. Hence, it is not realistic to require a
version number.

Kevin Kofler
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send
Adam Miller
2017-07-10 16:34:04 UTC
Permalink
Post by Kevin Kofler
Post by Adam Miller
In today's FESCo meeting we discussed the fact that there are many
RPMs currently in Fedora (a reported 244 in Rawhide currently) that
are defining a `Provides: bundled(<lib>) = <version>` but excluding
the version completely[0][1]. This removes that ability to properly
perform source code auditing and security vulnerability tracking.
My question to the Fedora Contributor Community is, how should we
handle this? Is this something that should just simply be fixed by the
packages currently violating the Guidelines, should the Guidelines be
altered in a way that makes this easier to deal with for Packagers but
also provides what is needed for auditing and vulnerability tracking,
or is there simply clarification needed by what is required in the
<version> field?
A version number may not even exist at all. Not all code that people copy is
a library with a version number. Copylibs often don't bother doing releases
because everyone just embeds it as a git submodule or checks out some random
revision to copy into their own SCM. Hence, it is not realistic to require a
version number.
So should we just stop requiring any RPMs be versioned since it's not
realistic to require a version number?

-AdamM
Post by Kevin Kofler
Kevin Kofler
_______________________________________________
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe s
Petr Pisar
2017-07-10 06:14:41 UTC
Permalink
Post by Adam Miller
`Provides: bundled(<lib>) = <version>` but excluding
the version completely[0][1]. This removes that ability to properly
perform source code auditing and security vulnerability tracking.
My question to the Fedora Contributor Community is, how should we
handle this?
It's wrong assumption that a version of the bundled code is known. Either
the bundled upstream has no versions, or the bundled code is heavily
modified (hence the reason for not unbundling and the need for Provides:
bundled()), or there the bundled code has no upstream (anymore).
I usually try to track the version when the code was bundled from, but
sometimes it's impossible to figure out.

Thus I recommend relaxing the guidelines. It's still better to have
an unversioned bundled() RPM symbol than nothing.

-- Petr
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-lea
Continue reading on narkive:
Loading...