Discussion:
Goodbye nvr.rsplit('-', 2), hello modularity
(too old to reply)
Patrick Uiterwijk
2018-03-15 15:14:34 UTC
Permalink
Hi all,

If you maintain any application in Fedora Infra (or outside) that
tries to parse things out of RPM names, you might be interested in
this.

For those wondering where I've been spending most of my time the last
few weeks: I've been deep in the internals of Bodhi fixing all kinds
of issues I've found between it and Modularity (turns out enabling
composing was just the beginning).

At some point, I was informed that for modules, they are throwing all
kinds of old assumptions out of the door.
One of those is the good old Name-Version-Release in RPM land.

Modules do not have an NVR, they have an NSVC: Name, Stream, Version, Context.
Name is the name of the module ("nodejs").
Stream is a stream, like "6", "7" or "8", these are crucial for the
parallel availability that modules are trying to accomplish.
Version is a date/time stamp, used to indicate new versions in the
same name-stream.
Context is a hash of some information in the built module that makes
it so that the nodejs-6-20170314 built for Fedora 28 has a different
identifier than the one built for Fedora 29.

There are some additional fields for installed things (arch and
profile), but those aren't really important for things trying to
parse/show module names I think.

One of the interesting things they wanted to also allow: dashes in streams.
As a consequence, when you get an N-S-V.C as modules are represented
in Koji builds, doing a .rsplit('-', 2) will not give you Name,
Stream, Version.Context per se.
You could totally have a module called
nodejs-my-stream-5-20170314.abcd, with name=nodejs,
stream=my-stream-5, version=20170314, context=abcd.
There is no way for you to independently figure out what the NSVC
components are, you will need to ask Koji, and use its name, version
and release fields (with name=name, version=stream,
release=version(.context)).

Also as a consequence, modules should not be dash-separated in
anything the users see.
For user consumption, they are Name:Stream:Version:Context, so you may
need to manually convert between one representation and the other if
you need to look up in koji or other systems versus display to users.

Also, note that if you are touching older modules in Koji (e.g.
because you go through all koji builds), modules before February 19th
will be lacking the Context field.
However, I've been told that those modules will not be used, so unless
you or your app goes spelunking in koji (for fun and profit), you
should never see those.

I hope that this is useful information for anyone else finding
themselves having to parse NVRs/NSVs.

Regards,
Patrick


p.s.: This is a forward of my email sent originally to the Infra list.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe sen
Mátyás Selmeci
2018-03-15 15:21:00 UTC
Permalink
Post by Patrick Uiterwijk
Hi all,
If you maintain any application in Fedora Infra (or outside) that
tries to parse things out of RPM names, you might be interested in
this.
For those wondering where I've been spending most of my time the last
few weeks: I've been deep in the internals of Bodhi fixing all kinds
of issues I've found between it and Modularity (turns out enabling
composing was just the beginning).
At some point, I was informed that for modules, they are throwing all
kinds of old assumptions out of the door.
One of those is the good old Name-Version-Release in RPM land.
Modules do not have an NVR, they have an NSVC: Name, Stream, Version, Context.
Name is the name of the module ("nodejs").
Stream is a stream, like "6", "7" or "8", these are crucial for the
parallel availability that modules are trying to accomplish.
Version is a date/time stamp, used to indicate new versions in the
same name-stream.
Context is a hash of some information in the built module that makes
it so that the nodejs-6-20170314 built for Fedora 28 has a different
identifier than the one built for Fedora 29.
There are some additional fields for installed things (arch and
profile), but those aren't really important for things trying to
parse/show module names I think.
One of the interesting things they wanted to also allow: dashes in streams.
As a consequence, when you get an N-S-V.C as modules are represented
in Koji builds, doing a .rsplit('-', 2) will not give you Name,
Stream, Version.Context per se.
You could totally have a module called
nodejs-my-stream-5-20170314.abcd, with name=nodejs,
stream=my-stream-5, version=20170314, context=abcd.
There is no way for you to independently figure out what the NSVC
components are, you will need to ask Koji, and use its name, version
and release fields (with name=name, version=stream,
release=version(.context)).
Also as a consequence, modules should not be dash-separated in
anything the users see.
For user consumption, they are Name:Stream:Version:Context, so you may
need to manually convert between one representation and the other if
you need to look up in koji or other systems versus display to users.
Also, note that if you are touching older modules in Koji (e.g.
because you go through all koji builds), modules before February 19th
will be lacking the Context field.
However, I've been told that those modules will not be used, so unless
you or your app goes spelunking in koji (for fun and profit), you
should never see those.
I hope that this is useful information for anyone else finding
themselves having to parse NVRs/NSVs.
Regards,
Patrick
This does not affect traditional RPM/SRPM packages, right?

Thanks,
-Mat
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email
Patrick Uiterwijk
2018-03-15 15:30:05 UTC
Permalink
Post by Mátyás Selmeci
This does not affect traditional RPM/SRPM packages, right?
Correct, this only impacts modules.
Post by Mátyás Selmeci
Thanks,
-Mat
_______________________________________________
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists
Tom Hughes
2018-03-15 15:40:48 UTC
Permalink
Post by Patrick Uiterwijk
Post by Mátyás Selmeci
This does not affect traditional RPM/SRPM packages, right?
Correct, this only impacts modules.
Well currently at any rate? What's the long term story?

Only when some module rpms accidentally leaked into rawhide
recently they definitely had the kind of rather unhelpful names
that you described in your post and I got the impression at
the time that it was anticipated they might come to rawhide
for real at some point.

Tom

--
Tom Hughes (***@compton.nu)
http://compton.nu/
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email
Patrick マルタインアンドレアス Uiterwijk
2018-03-15 15:56:32 UTC
Permalink
Post by Tom Hughes
Well currently at any rate? What's the long term story?
Only when some module rpms accidentally leaked into rawhide
recently they definitely had the kind of rather unhelpful names
that you described in your post and I got the impression at
the time that it was anticipated they might come to rawhide
for real at some point.
Note that the module builds I was talking about are the actual modules, not the module content.
The RPMs that ended up in rawhide standard repos (which was a bug, they should have gone to, and will go to a Modular side-repo), are standard RPMS with NVR like $packagename-$version-$release.module_$tag.
In other words, the packages that end up in the repos fit the NVR pattern, it's just that the release part has its .%{dist} replaced with .module_%{tag}.

My email was about module description builds as you'd submit them to Bodhi for example, like https://koji.fedoraproject.org/koji/buildinfo?buildID=1055963.
Post by Tom Hughes
Tom
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscr
Nico Kadel-Garcia
2018-03-15 15:56:07 UTC
Permalink
On Thu, Mar 15, 2018 at 11:14 AM, Patrick Uiterwijk
Post by Patrick Uiterwijk
Hi all,
If you maintain any application in Fedora Infra (or outside) that
tries to parse things out of RPM names, you might be interested in
this.
For those wondering where I've been spending most of my time the last
few weeks: I've been deep in the internals of Bodhi fixing all kinds
of issues I've found between it and Modularity (turns out enabling
composing was just the beginning).
At some point, I was informed that for modules, they are throwing all
kinds of old assumptions out of the door.
One of those is the good old Name-Version-Release in RPM land.
Modules do not have an NVR, they have an NSVC: Name, Stream, Version, Context.
Name is the name of the module ("nodejs").
Stream is a stream, like "6", "7" or "8", these are crucial for the
parallel availability that modules are trying to accomplish.
Version is a date/time stamp, used to indicate new versions in the
same name-stream.
Context is a hash of some information in the built module that makes
it so that the nodejs-6-20170314 built for Fedora 28 has a different
identifier than the one built for Fedora 29.
Isn't that what "%{dist}" is for? Do you have a pointer to where this
is laid out?

We've seen people try to re-invent version numbering before.with other
applications, to add features for their desired workflow. Examples
includes the hashes added to python module versions at pypi.org, which
seriously broke py2pack and forced them to add an impedance matching
utility on their website so the old tools continued to work. Is this a
really desirable change?
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe se
Daniel P. Berrangé
2018-03-15 16:19:46 UTC
Permalink
Post by Patrick Uiterwijk
Hi all,
If you maintain any application in Fedora Infra (or outside) that
tries to parse things out of RPM names, you might be interested in
this.
For those wondering where I've been spending most of my time the last
few weeks: I've been deep in the internals of Bodhi fixing all kinds
of issues I've found between it and Modularity (turns out enabling
composing was just the beginning).
At some point, I was informed that for modules, they are throwing all
kinds of old assumptions out of the door.
One of those is the good old Name-Version-Release in RPM land.
Modules do not have an NVR, they have an NSVC: Name, Stream, Version, Context.
Name is the name of the module ("nodejs").
Stream is a stream, like "6", "7" or "8", these are crucial for the
parallel availability that modules are trying to accomplish.
Version is a date/time stamp, used to indicate new versions in the
same name-stream.
Context is a hash of some information in the built module that makes
it so that the nodejs-6-20170314 built for Fedora 28 has a different
identifier than the one built for Fedora 29.
There are some additional fields for installed things (arch and
profile), but those aren't really important for things trying to
parse/show module names I think.
One of the interesting things they wanted to also allow: dashes in streams.
As a consequence, when you get an N-S-V.C as modules are represented
in Koji builds, doing a .rsplit('-', 2) will not give you Name,
Stream, Version.Context per se.
You could totally have a module called
nodejs-my-stream-5-20170314.abcd, with name=nodejs,
stream=my-stream-5, version=20170314, context=abcd.
There is no way for you to independently figure out what the NSVC
components are, you will need to ask Koji, and use its name, version
and release fields (with name=name, version=stream,
release=version(.context)).
Have a non-deterministic naming format whose ambiguity you can only
resolve by querying Koji feels like a pretty undesirable thing. At
least it doesn't leak into the RPMs themselves, but from a developer
pov it would be really nice to be able to unambiguously parse the
module names into their parts without calling out to RPC services.
There may be contexts where you want to do such things where you
would rather not have outbound network connectivity to anywhere,
let along Koji.

If we need to allow '-' in the streams, could we use something
other than '-' as the field separator for the module names to
remove ambiguity ?
Post by Patrick Uiterwijk
For user consumption, they are Name:Stream:Version:Context, so you may
need to manually convert between one representation and the other if
you need to look up in koji or other systems versus display to users.
What stops us using Name:Stream:Version:Context everywhere,
or Name:Stream:Context-Version or Name:Stream-Version-Context
if we need it closer to RPM NVR style ?


Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-leave
Chuck Anderson
2018-03-15 16:28:46 UTC
Permalink
Post by Daniel P. Berrangé
Post by Patrick Uiterwijk
For user consumption, they are Name:Stream:Version:Context, so you may
need to manually convert between one representation and the other if
you need to look up in koji or other systems versus display to users.
What stops us using Name:Stream:Version:Context everywhere,
or Name:Stream:Context-Version or Name:Stream-Version-Context
if we need it closer to RPM NVR style ?
: doesn't work very well in filenames due to it being a pathname separator in some filesystems among other things.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-l
Randy Barlow
2018-03-15 18:06:35 UTC
Permalink
Post by Chuck Anderson
: doesn't work very well in filenames due to it being a pathname separator in some filesystems among other things.
It sounds like this breaks a great deal of tooling. Can we reconsider
switching away from - separators in modules? Is allowing streams to have
-'s in them important enough to break so many tools?
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@list
Simo Sorce
2018-03-15 20:31:24 UTC
Permalink
Post by Randy Barlow
Post by Chuck Anderson
: doesn't work very well in filenames due to it being a pathname separator in some filesystems among other things.
It sounds like this breaks a great deal of tooling. Can we reconsider
switching away from - separators in modules? Is allowing streams to have
-'s in them important enough to break so many tools?
I vote for using ⨊ as the separator, makes things easier :-)

Simo.

--
Simo Sorce
Sr. Principal Software Engineer
Red Hat, Inc
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.or
langdon
2018-03-15 23:57:34 UTC
Permalink
Post by Simo Sorce
Post by Randy Barlow
Post by Chuck Anderson
: doesn't work very well in filenames due to it being a pathname separator in some filesystems among other things.
It sounds like this breaks a great deal of tooling. Can we reconsider
switching away from - separators in modules? Is allowing streams to have
-'s in them important enough to break so many tools?
I vote for using ⨊ as the separator, makes things easier :-)
Simo.
kidding aside, we are trying to get metadata forced in to the name
"field"[1] in to proper metadata fields rather than just mangling the
names. As a result, we end up with more fields and we don't have to
guess what they are based on conventions.

I also am not sure why we wouldn't want to have hyphens in the names of
modules. For example, a package for "react-native". We even say in the
naming guidelines "When naming packages for Fedora, the maintainer must
use the dash '-' as the delimiter for name parts." [2]. Although, this
is a bit unclear as to whether they mean "base name" or the overall rpm
name, from context, I think it is base name.

I could see an argument for using periods as the metadata separator,
similar to rpms. However, this might cause confusion with what the
elements of the metadata "mean" if they are not the same as (and in the
same order as) the elements of an rpm "name".

We chose a colon & slash (for profile names) somewhat based on the
practice(s) of docker. However, the dnf team made the ultimate choice,
so their opinion on the subject matters.

Langdon

[1]: what is referred to in the naming guidelines as the "base name" as
distinct from the full name of the rpm which would include N,V, & R

[2]:
https://fedoraproject.org/wiki/Packaging:Naming?rd=Packaging:NamingGuidelines

_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an emai
Adam Williamson
2018-03-16 00:47:41 UTC
Permalink
Post by langdon
Post by Simo Sorce
Post by Randy Barlow
Post by Chuck Anderson
: doesn't work very well in filenames due to it being a pathname separator in some filesystems among other things.
It sounds like this breaks a great deal of tooling. Can we reconsider
switching away from - separators in modules? Is allowing streams to have
-'s in them important enough to break so many tools?
I vote for using ⨊ as the separator, makes things easier :-)
Simo.
kidding aside, we are trying to get metadata forced in to the name
"field"[1] in to proper metadata fields rather than just mangling the
names. As a result, we end up with more fields and we don't have to
guess what they are based on conventions.
I also am not sure why we wouldn't want to have hyphens in the names of
modules. For example, a package for "react-native".
If you are trying to parse a delimited string, you can cope with either
the *first* or *last* field also being allowed to contain the
delimiter. This is how it works for RPM NEVRAs: name can contain a -,
but epoch, version, release and arch cannot. This allows reliable
parsing because you know any 'extra' -s must be part of the name;
practically speaking, as the subject of the thread suggests, you can
just do a .rsplit() (or equivalent in your language of choice) to get
the fields.

So: there would be no problem with - in the *name* of the modules, so
long as that's the first field, which apparently it is.

The problem is if you allow the delimiter in *both* the first and last
fields, or in *any other field besides the first and the last*. As soon
as you do that, parsing is impossible. This is the issue with module
names, as Patrick pointed out: they allow - in the stream, which is the
*second* field. As soon as you allow this, reliably parsing the name
components out of the whole name becomes impossible.

Yes, you can say 'just don't parse strings, go ask a service instead',
but parsing strings is sometimes by a long way the most convenient way
to do things, and I really can't see how allowing the delimiter
character to appear in one of the field names is a *bigger* win than
allowing the components to be reliably parsed from the overall name.

(I have extensive experience with this field, thanks to dealing with
RPM names, image filenames, *and* Pungi compose IDs (oh God, don't make
me break out my compose ID parser as an example), and at this point I'd
like to designate a special circle of hell for people who do this sort
of thing. :>)
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe s
Pierre-Yves Chibon
2018-03-16 09:18:25 UTC
Permalink
Post by Adam Williamson
So: there would be no problem with - in the *name* of the modules, so
long as that's the first field, which apparently it is.
The problem is if you allow the delimiter in *both* the first and last
fields, or in *any other field besides the first and the last*. As soon
as you do that, parsing is impossible. This is the issue with module
names, as Patrick pointed out: they allow - in the stream, which is the
*second* field. As soon as you allow this, reliably parsing the name
components out of the whole name becomes impossible.
Yes, you can say 'just don't parse strings, go ask a service instead',
but parsing strings is sometimes by a long way the most convenient way
to do things, and I really can't see how allowing the delimiter
character to appear in one of the field names is a *bigger* win than
allowing the components to be reliably parsed from the overall name.
I wonder what we can do about this. Is it FESCo material?
Can the folks working on modularity comment more on this?

This is quite failing the moto: it's ok to disagree, not to surprise


Pierre
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send
Randy Barlow
2018-03-16 19:40:34 UTC
Permalink
Post by Pierre-Yves Chibon
I wonder what we can do about this. Is it FESCo material?
Can the folks working on modularity comment more on this?
I am considering filing it for FESCo to consider. I haven't seen a
compelling explanation for why modules are using a different separator
than our other content types so far, and it really does break a lot of
tooling.

Koji also does not allow the :'s (so modules continue to use -'s there),
so it also causes an inconsistency in NSVC syntax for Koji and
everything else (which will be very confusing in the Bodhi UI, which
touches "everything else" as well as Koji).
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lis
Dennis Gregorovic
2018-03-16 20:04:04 UTC
Permalink
modules are not RPMs. I would not expect them to necessarily use the same
format as RPMs. If we take koji out of the equation, we have module builds
in N:S:V:C format and module RPMs in N-V-R.A format. They use different
separators, but both can be parsed consistently.

The challenge here is that those module builds are squeezed into a N-V-R
format when imported into Koji. The N-V-R format is used for all build
types in Koji and it works well for its purpose, but it's also not
realistic to expect that all content types are going to natively use N-V-R
outside of Koji. My suggestion is that we consider the N-V-R format of
modules to be a representation internal to koji and that N:S:V:C is the
format used to represent modules to users. If that means tweaking the Koji
UI we can look into that.

Cheers
-- Dennis
Post by Randy Barlow
Post by Pierre-Yves Chibon
I wonder what we can do about this. Is it FESCo material?
Can the folks working on modularity comment more on this?
I am considering filing it for FESCo to consider. I haven't seen a
compelling explanation for why modules are using a different separator
than our other content types so far, and it really does break a lot of
tooling.
Koji also does not allow the :'s (so modules continue to use -'s there),
so it also causes an inconsistency in NSVC syntax for Koji and
everything else (which will be very confusing in the Bodhi UI, which
touches "everything else" as well as Koji).
_______________________________________________
--
Dennis Gregorovic
Manager, PnT DevOps
Red Hat
***@redhat.com T: +1-978.392.3112 M: +1-617.901.9799
Randy Barlow
2018-03-16 20:14:27 UTC
Permalink
modules are not RPMs.  I would not expect them to necessarily use the
same format as RPMs.  If we take koji out of the equation, we have
module builds in N:S:V:C format and module RPMs in N-V-R.A format.  They
use different separators, but both can be parsed consistently. 
I don't contest the above, but I am arguing that it is needlessly
different and causes infrastructure software to need to handle module
strings differently than they handle RPMs and containers. Perhaps there
is a benefit that I've not been made aware of yet, but from where I sit
it seems like a change that causes problems without bringing a tangible
benefit.

The reasoning I've heard so far is that this allows stream names to have
-'s in them - is that important? I don't know it to be, but am open to
be convinced. Thus my question - is it important enough to have -'s in
stream names to justify the work needed to make all things that interact
with Koji parse them using a web service rather than local code (such as
rsplit('-', 2) in Python)?
If that means
tweaking the Koji UI we can look into that.
For my concerns, it would be more helpful to tweak the Koji API than the
UI, so that tools that interact with Koji (like Bodhi) don't have to
know that Modules are different in Koji than they are everywhere else.
Of course, I'd rather just having modules not use -'s in stream names as
I wrote above, but if we really are going that route it'd be ideal if we
used :'s everywhere, including Koji.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to deve
Adam Williamson
2018-03-16 22:21:06 UTC
Permalink
Post by Randy Barlow
Post by Dennis Gregorovic
If that means
tweaking the Koji UI we can look into that.
For my concerns, it would be more helpful to tweak the Koji API than the
UI, so that tools that interact with Koji (like Bodhi) don't have to
know that Modules are different in Koji than they are everywhere else.
Of course, I'd rather just having modules not use -'s in stream names as
I wrote above, but if we really are going that route it'd be ideal if we
used :'s everywhere, including Koji.
Thanks, that's what I was trying to get at. The problem is that Koji is
very big and very hairy and no-one enjoys poking it much.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel
Dennis Gregorovic
2018-03-17 00:28:08 UTC
Permalink
Don't be afraid of koji. :)

It is under active development. Of course, resources are limited and not
everything can be taken on, but it's worth having the discussion. I have
filed https://pagure.io/koji/issue/851 as a strawman and will try to
solicit feedback on the idea.

Patrick, if https://pagure.io/koji/issue/851 was implemented would that
resolve the challenge you were facing? If not, it might be worth reaching
out to koji-***@lists.fedorahosted.org and discussing it further there.

Cheers
-- Dennis
Post by Adam Williamson
Post by Randy Barlow
Post by Dennis Gregorovic
If that means
tweaking the Koji UI we can look into that.
For my concerns, it would be more helpful to tweak the Koji API than the
UI, so that tools that interact with Koji (like Bodhi) don't have to
know that Modules are different in Koji than they are everywhere else.
Of course, I'd rather just having modules not use -'s in stream names as
I wrote above, but if we really are going that route it'd be ideal if we
used :'s everywhere, including Koji.
Thanks, that's what I was trying to get at. The problem is that Koji is
very big and very hairy and no-one enjoys poking it much.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
--
Dennis Gregorovic
Manager, PnT DevOps
Red Hat
***@redhat.com T: +1-978.392.3112 M: +1-617.901.9799
Patrick マルタインアンドレアス Uiterwijk
2018-03-17 15:15:23 UTC
Permalink
Post by Dennis Gregorovic
Don't be afraid of koji. :)
I couldn't agree more, Koji's a pretty nice project to work on.
It might seems scary for newcomers because of all the ingrained knowledge and assumptions, but once you get close the community is nice and friendly :).
Post by Dennis Gregorovic
It is under active development. Of course, resources are limited and not
everything can be taken on, but it's worth having the discussion. I have
filed https://pagure.io/koji/issue/851 as a strawman and will try to
solicit feedback on the idea.
Thanks for doing that, though I don't know that it would resolve my challenges.
First, I'd like to point out that by now I've basically solved the problems for Bodhi and know how to deal with this change.
The main reason I wanted to send this email is because in things like fedmsg's and other things, the NVRs as imported by Koji still leak out.
Even if the identifiers are colon-separated in all places and the N-S-V encoded version is never exported out of koji, people will still need to update scripts and tooling, and bringing that to people's attention was my main reason for sending the email.

I absolutely did not mean for my email to be judgemental, as I've not been privy to the decision to allow dashes in multiple fields or the rest, so I can't really judge the reasoning or the conclusion.
If it came over as judging, I'm sorry, I really just wanted to give people a headsup so that they have some upfront time to update scripts before seeing eithe module identifiers in fedmsg's or any other place.
Post by Dennis Gregorovic
Patrick, if https://pagure.io/koji/issue/851 was implemented would that
resolve the challenge you were facing? If not, it might be worth reaching
out to koji-devel(a)lists.fedorahosted.org and discussing it further there.
It would not, but that's okay.
It would still mean that Module identifiers, whether you notice them as N-S-V.C or N:S:V:C, are new, and all tooling and scripts will need to be updated to handle those new identifiers.
Post by Dennis Gregorovic
Cheers
-- Dennis
Patrick
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an emai
Adam Williamson
2018-03-17 15:45:37 UTC
Permalink
On Sat, 2018-03-17 at 15:15 +0000, Patrick マルタインアンドレアス Uiterwijk
Post by Patrick マルタインアンドレアス Uiterwijk
Post by Dennis Gregorovic
Don't be afraid of koji. :)
I couldn't agree more, Koji's a pretty nice project to work on.
It might seems scary for newcomers because of all the ingrained
knowledge and assumptions, but once you get close the community is
nice and friendly :).
Sorry, I should've been clearer - I wasn't referring to any of these
attributes, but just to how important Koji is and how many different
key processes it involves. This makes it inherently complex to change
significant things in it because there are just so many other systems
and processes hooked into Koji and it's very difficult to know how
they're all going to react.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email
Jan Kaluza
2018-03-19 09:10:23 UTC
Permalink
On Sat, Mar 17, 2018 at 4:15 PM, Patrick マルタむンアンドレアス Uiterwijk <
Post by Patrick マルタインアンドレアス Uiterwijk
Post by Dennis Gregorovic
Don't be afraid of koji. :)
I couldn't agree more, Koji's a pretty nice project to work on.
It might seems scary for newcomers because of all the ingrained knowledge
and assumptions, but once you get close the community is nice and friendly
:).
Post by Dennis Gregorovic
It is under active development. Of course, resources are limited and not
everything can be taken on, but it's worth having the discussion. I have
filed https://pagure.io/koji/issue/851 as a strawman and will try to
solicit feedback on the idea.
Thanks for doing that, though I don't know that it would resolve my challenges.
First, I'd like to point out that by now I've basically solved the
problems for Bodhi and know how to deal with this change.
The main reason I wanted to send this email is because in things like
fedmsg's and other things, the NVRs as imported by Koji still leak out.
Even if the identifiers are colon-separated in all places and the N-S-V
encoded version is never exported out of koji, people will still need to
update scripts and tooling, and bringing that to people's attention was my
main reason for sending the email.
I absolutely did not mean for my email to be judgemental, as I've not been
privy to the decision to allow dashes in multiple fields or the rest, so I
can't really judge the reasoning or the conclusion.
If it came over as judging, I'm sorry, I really just wanted to give people
a headsup so that they have some upfront time to update scripts before
seeing eithe module identifiers in fedmsg's or any other place.
Post by Dennis Gregorovic
Patrick, if https://pagure.io/koji/issue/851 was implemented would that
resolve the challenge you were facing? If not, it might be worth
reaching
Post by Dennis Gregorovic
out to koji-devel(a)lists.fedorahosted.org and discussing it further
there.
It would not, but that's okay.
It would still mean that Module identifiers, whether you notice them as
N-S-V.C or N:S:V:C, are new, and all tooling and scripts will need to be
updated to handle those new identifiers.
I've read some parts of this thread and I see people are kind of saying "we
have to update all the tooling". I really don't think there is tooling
which needs to be updated *just* because of the delimiter.

If something does not support modules and it should support modules and
*needs* to work with Koji content-generator imported builds for some
reason, it needs to updated to support modules and as part of this update,
it can be told to use ':' as its output.

If something does support modules already, it needs to be updated to
support "context" field anyway and as part of this update ... ^

Note that this is just about the "module" type koji builds imported to Koji
by MBS once the module is built. I think that t if you have no idea that
those builds exist, you have nothing to update in your tool.

What is impacted by this is:
- Bodhi - which is easy to update and I've sent a PR which has been
continued on by Patrick and is merged now.
- Pungi - this supports ":" already.
- MBS, pdc-updater - supports ":" already.

Is there actually anything else?

Regards,
Jan Kaluza
Post by Patrick マルタインアンドレアス Uiterwijk
Post by Dennis Gregorovic
Cheers
-- Dennis
Patrick
_______________________________________________
Pierre-Yves Chibon
2018-03-19 10:52:31 UTC
Permalink
This post might be inappropriate. Click to display it.
Zbigniew Jędrzejewski-Szmek
2018-03-18 19:09:23 UTC
Permalink
Post by Randy Barlow
modules are not RPMs.  I would not expect them to necessarily use the
same format as RPMs.  If we take koji out of the equation, we have
module builds in N:S:V:C format and module RPMs in N-V-R.A format.  They
use different separators, but both can be parsed consistently. 
I don't contest the above, but I am arguing that it is needlessly
different and causes infrastructure software to need to handle module
strings differently than they handle RPMs and containers. Perhaps there
is a benefit that I've not been made aware of yet, but from where I sit
it seems like a change that causes problems without bringing a tangible
benefit.
The reasoning I've heard so far is that this allows stream names to have
-'s in them - is that important? I don't know it to be, but am open to
be convinced. Thus my question - is it important enough to have -'s in
stream names to justify the work needed to make all things that interact
with Koji parse them using a web service rather than local code (such as
rsplit('-', 2) in Python)?
So, is anyone knowledgeable who can answer this question?

If nobody can, then mostly likely the answer is "no, it's not important".

Zbyszek
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email
Petr Šabata
2018-03-19 08:27:30 UTC
Permalink
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
modules are not RPMs.  I would not expect them to necessarily use the
same format as RPMs.  If we take koji out of the equation, we have
module builds in N:S:V:C format and module RPMs in N-V-R.A format.  They
use different separators, but both can be parsed consistently. 
I don't contest the above, but I am arguing that it is needlessly
different and causes infrastructure software to need to handle module
strings differently than they handle RPMs and containers. Perhaps there
is a benefit that I've not been made aware of yet, but from where I sit
it seems like a change that causes problems without bringing a tangible
benefit.
The reasoning I've heard so far is that this allows stream names to have
-'s in them - is that important? I don't know it to be, but am open to
be convinced. Thus my question - is it important enough to have -'s in
stream names to justify the work needed to make all things that interact
with Koji parse them using a web service rather than local code (such as
rsplit('-', 2) in Python)?
So, is anyone knowledgeable who can answer this question?
If nobody can, then mostly likely the answer is "no, it's not important".
The reason for allowing hyphens was to, maybe ironically, allow
more flexibility and to avoid unnecessary restriction on names.

While streams are often presented as upstream versions in
most of the examples, they're strings and are, in reality, an
extension to the name. Your stream might be just "6" or "8" but
it could also be "experimental-build" or "performance-patches".
Current tooling shortcomings shouldn't influence how details
like this are designed. The tooling can be fixed (or enhanced,
if you prefer).

The ordering of the fields is based on the importance of
the field to the user and most of them are optional when
people interact with modules using their package manager.
"module", "module:stream", "module:stream:version" are all
valid inputs (we do not expect people to use context directly;
it's just serialized that distinguishes individual builds for
stream expansion). Context is indeed similar to dist-tag,
just richer. Given the parallel availability, your modules
aren't built just for a release of Fedora, they are built for
any valid combination of their modular dependncies, with the
Fedora release being just one of them.

Note, as Patrick's already said, these names do not leak
anywhere. Modules are not RPMs and are not handled as such.
This, AFAIK, only affects what you see in some parts of the
build and update system (and on the bus).

Also note that generally if you have access to modules, you
have access to their (structured) metadata.

P
Adam Samalik
2018-03-19 08:53:47 UTC
Permalink
(a repost from the infra list)


Please note that modules (and therefore module builds) may reference to
multiple RPM builds. Also, the module name doesn't need to match any RPM
name. Therefore getting package names from the module build name would not
be possible anyway.


As an exapmple, see the following build: https://koji.fedoraproject.
org/koji/buildinfo?buildID=1057280

Build name: "golang-ecosystem-2017.0-20180312141905.40a0ab3e"

If you open the modulemd [1] (a module definition file) linked at the
bottom of the page, you'll see that this module build contains four RPM
components (SRPMs):
* golang-github-russross-blackfriday
* go-srpm-macros
* golang-github-cpuguy83-go-md2man
* go-compilers

The page also says "Package Name golang-ecosystem" which links to a
"golang-ecosystem" package [2] in Koji. That's weird, there is no such
package named "golang-ecosystem". The page lists two module builds, not two
package builds.


Another example, where a module references only one RPM package and the
module name matches the package name: https://koji.fedoraproject.
org/koji/buildinfo?buildID=1055922

Build name: "nodejs-6-20180308155546.c2c572ec"

Here, the modulemd [3] lists just one RPM component (SRPM):
* nodejs

The page also says "Package Name nodejs" which again links to a "nodejs"
package [4] in Koji. But this time, there *is* a "nodejs" package as well
as a "nodejs" module. That page lists both!

An example of a "nodejs" package [5], build name
"nodejs-9.8.0-1.module_1571+4f4bc63d" which is a good old NVR.

An example of a "nodejs" module [6], build name
"nodejs-6-20180308155546.c2c572ec"
whic is not an NVR, it's a NSVC (name, stream, version, context).


So after going through these two examples, I think this discussion should
be more about:
1. storing two completely different IDs (NVR and NSVC) in the same data
structure
2. listing two different build types (package and module) in the same
place, both as package builds



[1] https://kojipkgs.fedoraproject.org//packages/golang-ecosystem/2017.0/
20180312141905.40a0ab3e/files/module/modulemd.txt
[2] https://koji.fedoraproject.org/koji/packageinfo?packageID=25886
[3] https://kojipkgs.fedoraproject.org//packages/nodejs/6/20180308155546.
c2c572ec/files/module/modulemd.txt
[4] https://koji.fedoraproject.org/koji/packageinfo?packageID=15154
[5] https://koji.fedoraproject.org/koji/buildinfo?buildID=1055873
[6] https://koji.fedoraproject.org/koji/buildinfo?buildID=1055927
Post by Randy Barlow
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
Post by Dennis Gregorovic
modules are not RPMs. I would not expect them to necessarily use the
same format as RPMs. If we take koji out of the equation, we have
module builds in N:S:V:C format and module RPMs in N-V-R.A format.
They
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
Post by Dennis Gregorovic
use different separators, but both can be parsed consistently.
I don't contest the above, but I am arguing that it is needlessly
different and causes infrastructure software to need to handle module
strings differently than they handle RPMs and containers. Perhaps there
is a benefit that I've not been made aware of yet, but from where I sit
it seems like a change that causes problems without bringing a tangible
benefit.
The reasoning I've heard so far is that this allows stream names to
have
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
-'s in them - is that important? I don't know it to be, but am open to
be convinced. Thus my question - is it important enough to have -'s in
stream names to justify the work needed to make all things that
interact
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
with Koji parse them using a web service rather than local code (such
as
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
rsplit('-', 2) in Python)?
So, is anyone knowledgeable who can answer this question?
If nobody can, then mostly likely the answer is "no, it's not important".
The reason for allowing hyphens was to, maybe ironically, allow
more flexibility and to avoid unnecessary restriction on names.
While streams are often presented as upstream versions in
most of the examples, they're strings and are, in reality, an
extension to the name. Your stream might be just "6" or "8" but
it could also be "experimental-build" or "performance-patches".
Current tooling shortcomings shouldn't influence how details
like this are designed. The tooling can be fixed (or enhanced,
if you prefer).
The ordering of the fields is based on the importance of
the field to the user and most of them are optional when
people interact with modules using their package manager.
"module", "module:stream", "module:stream:version" are all
valid inputs (we do not expect people to use context directly;
it's just serialized that distinguishes individual builds for
stream expansion). Context is indeed similar to dist-tag,
just richer. Given the parallel availability, your modules
aren't built just for a release of Fedora, they are built for
any valid combination of their modular dependncies, with the
Fedora release being just one of them.
Note, as Patrick's already said, these names do not leak
anywhere. Modules are not RPMs and are not handled as such.
This, AFAIK, only affects what you see in some parts of the
build and update system (and on the bus).
Also note that generally if you have access to modules, you
have access to their (structured) metadata.
P
_______________________________________________
--
Adam Šamalík
---------------------------
Software Engineer
Red Hat
Zbigniew Jędrzejewski-Szmek
2018-03-19 08:57:08 UTC
Permalink
Post by Petr Å abata
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
modules are not RPMs.  I would not expect them to necessarily use the
same format as RPMs.  If we take koji out of the equation, we have
module builds in N:S:V:C format and module RPMs in N-V-R.A format.  They
use different separators, but both can be parsed consistently. 
I don't contest the above, but I am arguing that it is needlessly
different and causes infrastructure software to need to handle module
strings differently than they handle RPMs and containers. Perhaps there
is a benefit that I've not been made aware of yet, but from where I sit
it seems like a change that causes problems without bringing a tangible
benefit.
The reasoning I've heard so far is that this allows stream names to have
-'s in them - is that important? I don't know it to be, but am open to
be convinced. Thus my question - is it important enough to have -'s in
stream names to justify the work needed to make all things that interact
with Koji parse them using a web service rather than local code (such as
rsplit('-', 2) in Python)?
So, is anyone knowledgeable who can answer this question?
If nobody can, then mostly likely the answer is "no, it's not important".
The reason for allowing hyphens was to, maybe ironically, allow
more flexibility and to avoid unnecessary restriction on names.
While streams are often presented as upstream versions in
most of the examples, they're strings and are, in reality, an
extension to the name. Your stream might be just "6" or "8" but
it could also be "experimental-build" or "performance-patches".
Current tooling shortcomings shouldn't influence how details
like this are designed. The tooling can be fixed (or enhanced,
if you prefer).
Thank you for the detailed reply.

I see the reasons, and the elegance of not placing restrictions on the
stream field, but I think that's one of those where practicality beats
purity. _Tooling_ may be improved, but this is also about _humans_.
With the current format, it is simply impossible to look at a module
n-s-v-c string and parse it into components. In other words, it's not
possible to say what the *name* is. This is a major practical issue.

Frankly, I don't think it's a big restriction to forbid dashes in
stream name. Colons, semicolons, slashes, spaces, and a host of other
characters have to be forbidden too. If we establish a convention to
use plus, underscore, dot, or whatever is the most appropriate, in
a few months people will not remember why they ever wanted dashes.

Zbyszek
Post by Petr Å abata
The ordering of the fields is based on the importance of
the field to the user and most of them are optional when
people interact with modules using their package manager.
"module", "module:stream", "module:stream:version" are all
valid inputs (we do not expect people to use context directly;
it's just serialized that distinguishes individual builds for
stream expansion). Context is indeed similar to dist-tag,
just richer. Given the parallel availability, your modules
aren't built just for a release of Fedora, they are built for
any valid combination of their modular dependncies, with the
Fedora release being just one of them.
Note, as Patrick's already said, these names do not leak
anywhere. Modules are not RPMs and are not handled as such.
This, AFAIK, only affects what you see in some parts of the
build and update system (and on the bus).
Also note that generally if you have access to modules, you
have access to their (structured) metadata.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email
Petr Šabata
2018-03-19 09:18:13 UTC
Permalink
Post by Zbigniew Jędrzejewski-Szmek
Post by Petr Å abata
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
modules are not RPMs.  I would not expect them to necessarily use the
same format as RPMs.  If we take koji out of the equation, we have
module builds in N:S:V:C format and module RPMs in N-V-R.A format.  They
use different separators, but both can be parsed consistently. 
I don't contest the above, but I am arguing that it is needlessly
different and causes infrastructure software to need to handle module
strings differently than they handle RPMs and containers. Perhaps there
is a benefit that I've not been made aware of yet, but from where I sit
it seems like a change that causes problems without bringing a tangible
benefit.
The reasoning I've heard so far is that this allows stream names to have
-'s in them - is that important? I don't know it to be, but am open to
be convinced. Thus my question - is it important enough to have -'s in
stream names to justify the work needed to make all things that interact
with Koji parse them using a web service rather than local code (such as
rsplit('-', 2) in Python)?
So, is anyone knowledgeable who can answer this question?
If nobody can, then mostly likely the answer is "no, it's not important".
The reason for allowing hyphens was to, maybe ironically, allow
more flexibility and to avoid unnecessary restriction on names.
While streams are often presented as upstream versions in
most of the examples, they're strings and are, in reality, an
extension to the name. Your stream might be just "6" or "8" but
it could also be "experimental-build" or "performance-patches".
Current tooling shortcomings shouldn't influence how details
like this are designed. The tooling can be fixed (or enhanced,
if you prefer).
Thank you for the detailed reply.
I see the reasons, and the elegance of not placing restrictions on the
stream field, but I think that's one of those where practicality beats
purity. _Tooling_ may be improved, but this is also about _humans_.
With the current format, it is simply impossible to look at a module
n-s-v-c string and parse it into components. In other words, it's not
possible to say what the *name* is. This is a major practical issue.
We would really prefer people looking at the structured data
rather than the imperfect string when processing modules.
Modules carry quite a lot of it; strings are just identifiers.
I know it's tempting but doing so might produce suboptimal
results. That said, if people here (or FESCo) feel like decoding
ID strings is crucial, another policy limiting the options in
Fedora might always be added. I'd say it'd be unfortunate, though.
Post by Zbigniew Jędrzejewski-Szmek
Frankly, I don't think it's a big restriction to forbid dashes in
stream name. Colons, semicolons, slashes, spaces, and a host of other
characters have to be forbidden too. If we establish a convention to
use plus, underscore, dot, or whatever is the most appropriate, in
a few months people will not remember why they ever wanted dashes.
Yes, but unlike all of the examples above, dashes are
comparatively more common as word delimeters in this context and
I think is the most natural choice. New rules and conventions
can be defined, of course, but it's still forcing people to
work around ambiguous technological limitations.

P
Post by Zbigniew Jędrzejewski-Szmek
Zbyszek
Post by Petr Å abata
The ordering of the fields is based on the importance of
the field to the user and most of them are optional when
people interact with modules using their package manager.
"module", "module:stream", "module:stream:version" are all
valid inputs (we do not expect people to use context directly;
it's just serialized that distinguishes individual builds for
stream expansion). Context is indeed similar to dist-tag,
just richer. Given the parallel availability, your modules
aren't built just for a release of Fedora, they are built for
any valid combination of their modular dependncies, with the
Fedora release being just one of them.
Note, as Patrick's already said, these names do not leak
anywhere. Modules are not RPMs and are not handled as such.
This, AFAIK, only affects what you see in some parts of the
build and update system (and on the bus).
Also note that generally if you have access to modules, you
have access to their (structured) metadata.
_______________________________________________
Fabio Valentini
2018-03-19 09:38:58 UTC
Permalink
Post by langdon
Post by Zbigniew Jędrzejewski-Szmek
On Sun, Mar 18, 2018 at 07:09:23PM +0000, Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
Post by Dennis Gregorovic
modules are not RPMs. I would not expect them to necessarily
use the
Post by Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
Post by Dennis Gregorovic
same format as RPMs. If we take koji out of the equation, we
have
Post by Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
Post by Dennis Gregorovic
module builds in N:S:V:C format and module RPMs in N-V-R.A
format. They
Post by Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
Post by Dennis Gregorovic
use different separators, but both can be parsed consistently.
I don't contest the above, but I am arguing that it is needlessly
different and causes infrastructure software to need to handle
module
Post by Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
strings differently than they handle RPMs and containers. Perhaps
there
Post by Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
is a benefit that I've not been made aware of yet, but from where
I sit
Post by Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
it seems like a change that causes problems without bringing a
tangible
Post by Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
benefit.
The reasoning I've heard so far is that this allows stream names
to have
Post by Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
-'s in them - is that important? I don't know it to be, but am
open to
Post by Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
be convinced. Thus my question - is it important enough to have
-'s in
Post by Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
stream names to justify the work needed to make all things that
interact
Post by Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
with Koji parse them using a web service rather than local code
(such as
Post by Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Randy Barlow
rsplit('-', 2) in Python)?
So, is anyone knowledgeable who can answer this question?
If nobody can, then mostly likely the answer is "no, it's not
important".
Post by Zbigniew Jędrzejewski-Szmek
The reason for allowing hyphens was to, maybe ironically, allow
more flexibility and to avoid unnecessary restriction on names.
While streams are often presented as upstream versions in
most of the examples, they're strings and are, in reality, an
extension to the name. Your stream might be just "6" or "8" but
it could also be "experimental-build" or "performance-patches".
Current tooling shortcomings shouldn't influence how details
like this are designed. The tooling can be fixed (or enhanced,
if you prefer).
Thank you for the detailed reply.
I see the reasons, and the elegance of not placing restrictions on the
stream field, but I think that's one of those where practicality beats
purity. _Tooling_ may be improved, but this is also about _humans_.
With the current format, it is simply impossible to look at a module
n-s-v-c string and parse it into components. In other words, it's not
possible to say what the *name* is. This is a major practical issue.
We would really prefer people looking at the structured data
rather than the imperfect string when processing modules.
Modules carry quite a lot of it; strings are just identifiers.
I know it's tempting but doing so might produce suboptimal
results. That said, if people here (or FESCo) feel like decoding
ID strings is crucial, another policy limiting the options in
Fedora might always be added. I'd say it'd be unfortunate, though.
Post by Zbigniew Jędrzejewski-Szmek
Frankly, I don't think it's a big restriction to forbid dashes in
stream name. Colons, semicolons, slashes, spaces, and a host of other
characters have to be forbidden too. If we establish a convention to
use plus, underscore, dot, or whatever is the most appropriate, in
a few months people will not remember why they ever wanted dashes.
Yes, but unlike all of the examples above, dashes are
comparatively more common as word delimeters in this context and
I think is the most natural choice. New rules and conventions
can be defined, of course, but it's still forcing people to
work around ambiguous technological limitations.
P
So you think having to send a request to a web service instead of just
parsing a string locally with one line of code is a good trade-off for
allowing dashes?

Gaining one additional allowed character and therefore losing the ability
to parse this locally just seems mad to me ... compared to that,
introducing a "convention" of using underscores (or sth. else) as word
delimiters within fields looks more reasonable by multiple orders of
magnitude ...

Just my 2c.

Fabio
Post by langdon
Zbyszek
Post by Zbigniew Jędrzejewski-Szmek
The ordering of the fields is based on the importance of
the field to the user and most of them are optional when
people interact with modules using their package manager.
"module", "module:stream", "module:stream:version" are all
valid inputs (we do not expect people to use context directly;
it's just serialized that distinguishes individual builds for
stream expansion). Context is indeed similar to dist-tag,
just richer. Given the parallel availability, your modules
aren't built just for a release of Fedora, they are built for
any valid combination of their modular dependncies, with the
Fedora release being just one of them.
Note, as Patrick's already said, these names do not leak
anywhere. Modules are not RPMs and are not handled as such.
This, AFAIK, only affects what you see in some parts of the
build and update system (and on the bus).
Also note that generally if you have access to modules, you
have access to their (structured) metadata.
_______________________________________________
_______________________________________________
Stanislav Ochotnicky
2018-03-19 09:58:08 UTC
Permalink
Post by Fabio Valentini
So you think having to send a request to a web service instead of just
parsing a string locally with one line of code is a good trade-off for
allowing dashes?
This has been mentioned several times in this thread and I think there's
a misunderstanding around this.

So: When your tool/whatever works with modules it will have to have
module metadata available in some form. In most client-side tools, I'd
guess that will be modulemd metadata in dnf repositories that will get
synced locally (just like rpm/yum repodata). Or it really might be
you'll query koji for the metadata if needed on a system without local
dnf metadata. But if you're working on top of module repositories -
metadata will be there.

RPMs *in* modules still use "N-V-R.A" strings and you can keep parsing
them. That is not changing. But the module that contains them uses
different strategy that allows maintainers more flexibility. And there
does not have to be exact match between module/stream etc name and NVRs
of rpms inside as others have mentioned.


--
Stanislav Ochotnicky <***@redhat.com>
Software Engineer, PnT DevOps - Brno

PGP: F434 2286 27DC 7D9B 2B64 0866 BCBD 752E 7B08 7241
Red Hat Inc. http://www.redhat.com
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.
Adam Williamson
2018-03-19 15:43:47 UTC
Permalink
Post by Stanislav Ochotnicky
Post by Fabio Valentini
So you think having to send a request to a web service instead of just
parsing a string locally with one line of code is a good trade-off for
allowing dashes?
This has been mentioned several times in this thread and I think there's
a misunderstanding around this.
So: When your tool/whatever works with modules it will have to have
module metadata available in some form.
What if I don't want to work with modules at all, just with information
about modules?

What if I just want to write a tool that parses fedmsgs about module
builds in Koji and does something that depends on module names, streams
and versions? (e.g. some sort of changelog thing)
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscri
Nils Philippsen
2018-03-19 16:18:13 UTC
Permalink
Post by Adam Williamson
What if I just want to write a tool that parses fedmsgs about module
builds in Koji and does something that depends on module names, streams
and versions? (e.g. some sort of changelog thing)
I haven't looked at actual fedmsgs lately but they should carry this
information as separate fields, e.g. here's how a module state change
message is specified:

https://fedora-fedmsg.readthedocs.io/en/latest/topics.html#mbs-module-state-change

The `context` field is missing here but I guess that's just a case of
the docs not keeping up with the latest changes.

Nils
--
Nils Philippsen "Those who would give up Essential Liberty to
Software Engineer purchase a little Temporary Safety, deserve neither
Red Hat Liberty nor Safety." -- Benjamin Franklin, 1759
PGP fingerprint: C4A8 9474 5C4C ADE3 2B8F 656D 47D8 9B65 6951 3011
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fe
Adam Williamson
2018-03-19 16:36:41 UTC
Permalink
Post by Nils Philippsen
Post by Adam Williamson
What if I just want to write a tool that parses fedmsgs about module
builds in Koji and does something that depends on module names, streams
and versions? (e.g. some sort of changelog thing)
I haven't looked at actual fedmsgs lately but they should carry this
information as separate fields, e.g. here's how a module state change
https://fedora-fedmsg.readthedocs.io/en/latest/topics.html#mbs-module-state-change
The `context` field is missing here but I guess that's just a case of
the docs not keeping up with the latest changes.
Ah, that is useful. Thanks.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send
Jan Kaluza
2018-03-19 19:08:29 UTC
Permalink
Post by Adam Williamson
Post by Stanislav Ochotnicky
Post by Fabio Valentini
So you think having to send a request to a web service instead of just
parsing a string locally with one line of code is a good trade-off for
allowing dashes?
This has been mentioned several times in this thread and I think there's
a misunderstanding around this.
So: When your tool/whatever works with modules it will have to have
module metadata available in some form.
What if I don't want to work with modules at all, just with information
about modules?
What if I just want to write a tool that parses fedmsgs about module
builds in Koji and does something that depends on module names, streams
and versions? (e.g. some sort of changelog thing)
Your tool can use good old buildsys.build.state.change fedmsg with the well
known name/version/release fields like this:

https://apps.stg.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.stg.buildsys.build.state.change

That's staging, because it's easier for me to find the module builds there
:).

Unfortunatelly, Koji does not add "type" field there, so you cannot find
out if that's module or not. But in case your tool is doing general
changelog per Koji build, it will work quite well with modules without you
even noticing you are working with modules.

In case of tools, you can even query koji using its api and pass the
name/version/release there, instead of NVR.

If you query Koji for the module build, you will actually find whole module
metadata in "extra" part of the build.

In case you are OK with using MBS fedmsgs, you can use the fedmsgs Nils
already sent in this thread.

Regards,
Jan Kaluza

--
Post by Adam Williamson
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
Michal Novotny
2018-03-20 09:06:30 UTC
Permalink
The question here is why don't we actually implement the modularity
the simple way.

Let's suppose I would like to make nodejs-8 available in EPEL7.

The most effortless way would be to create subdirectory called 8 at
src.fedoraproject.org/rpms/nodejs
with the updated spec file and sources for the bumped major version of
the nodejs package.

For fedpkg to be able to upload and download sources for that
subdirectory (submodule), the attached
single-line patch is needed in python-rpkg. There are a few more
changes needed to make srpm
importing work but it should be basically almost done as well.

Koji then just needs to learn how to build DistGit repositories
containing submodules where
submodule is any subdirectory containing a spec file. When Koji
discovers all the submodules
in a repository, it can start an srpm and a consequent rpm build for
each of them.

It is a very similar approach to what find_packages from python
setuptools implements. It
recognizes subpackages based on the presence of __init__.py file. The
only difference here
is that we will be recognizing subpackages (=submodules) by the
presence of *.spec file.

Everything else should be pretty much the same otherwise. Koji builds
nodejs-6 and nodejs-8 rpms
both with el7 disttag and both of these can go through the Bodhi
update process separately.

Of course, the question is if you can actually build and run nodejs-8
in EL7 successfully but
I think that will always be a problem no matter what the implementation is.

clime
On Mon, Mar 19, 2018 at 4:43 PM, Adam Williamson
Post by Adam Williamson
Post by Stanislav Ochotnicky
Post by Fabio Valentini
So you think having to send a request to a web service instead of just
parsing a string locally with one line of code is a good trade-off for
allowing dashes?
This has been mentioned several times in this thread and I think there's
a misunderstanding around this.
So: When your tool/whatever works with modules it will have to have
module metadata available in some form.
What if I don't want to work with modules at all, just with information
about modules?
What if I just want to write a tool that parses fedmsgs about module
builds in Koji and does something that depends on module names, streams
and versions? (e.g. some sort of changelog thing)
Your tool can use good old buildsys.build.state.change fedmsg with the well
https://apps.stg.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.stg.buildsys.build.state.change
That's staging, because it's easier for me to find the module builds there
:).
Unfortunatelly, Koji does not add "type" field there, so you cannot find out
if that's module or not. But in case your tool is doing general changelog
per Koji build, it will work quite well with modules without you even
noticing you are working with modules.
In case of tools, you can even query koji using its api and pass the
name/version/release there, instead of NVR.
If you query Koji for the module build, you will actually find whole module
metadata in "extra" part of the build.
In case you are OK with using MBS fedmsgs, you can use the fedmsgs Nils
already sent in this thread.
Regards,
Jan Kaluza
Post by Adam Williamson
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
_______________________________________________
Jan Kaluza
2018-03-20 14:31:53 UTC
Permalink
Post by Michal Novotny
The question here is why don't we actually implement the modularity
the simple way.
Let's suppose I would like to make nodejs-8 available in EPEL7.
The most effortless way would be to create subdirectory called 8 at
src.fedoraproject.org/rpms/nodejs
with the updated spec file and sources for the bumped major version of
the nodejs package.
What if your module contains packages coming from multiple SRPMs and they
depend on each other?

Jan Kaluza
Post by Michal Novotny
For fedpkg to be able to upload and download sources for that
subdirectory (submodule), the attached
single-line patch is needed in python-rpkg. There are a few more
changes needed to make srpm
importing work but it should be basically almost done as well.
Koji then just needs to learn how to build DistGit repositories
containing submodules where
submodule is any subdirectory containing a spec file. When Koji
discovers all the submodules
in a repository, it can start an srpm and a consequent rpm build for
each of them.
It is a very similar approach to what find_packages from python
setuptools implements. It
recognizes subpackages based on the presence of __init__.py file. The
only difference here
is that we will be recognizing subpackages (=submodules) by the
presence of *.spec file.
Everything else should be pretty much the same otherwise. Koji builds
nodejs-6 and nodejs-8 rpms
both with el7 disttag and both of these can go through the Bodhi
update process separately.
Of course, the question is if you can actually build and run nodejs-8
in EL7 successfully but
I think that will always be a problem no matter what the implementation is.
clime
On Mon, Mar 19, 2018 at 4:43 PM, Adam Williamson
Post by Adam Williamson
Post by Stanislav Ochotnicky
Post by Fabio Valentini
So you think having to send a request to a web service instead of
just
Post by Adam Williamson
Post by Stanislav Ochotnicky
Post by Fabio Valentini
parsing a string locally with one line of code is a good trade-off
for
Post by Adam Williamson
Post by Stanislav Ochotnicky
Post by Fabio Valentini
allowing dashes?
This has been mentioned several times in this thread and I think
there's
Post by Adam Williamson
Post by Stanislav Ochotnicky
a misunderstanding around this.
So: When your tool/whatever works with modules it will have to have
module metadata available in some form.
What if I don't want to work with modules at all, just with information
about modules?
What if I just want to write a tool that parses fedmsgs about module
builds in Koji and does something that depends on module names, streams
and versions? (e.g. some sort of changelog thing)
Your tool can use good old buildsys.build.state.change fedmsg with the
well
https://apps.stg.fedoraproject.org/datagrepper/
raw?topic=org.fedoraproject.stg.buildsys.build.state.change
That's staging, because it's easier for me to find the module builds
there
:).
Unfortunatelly, Koji does not add "type" field there, so you cannot find
out
if that's module or not. But in case your tool is doing general changelog
per Koji build, it will work quite well with modules without you even
noticing you are working with modules.
In case of tools, you can even query koji using its api and pass the
name/version/release there, instead of NVR.
If you query Koji for the module build, you will actually find whole
module
metadata in "extra" part of the build.
In case you are OK with using MBS fedmsgs, you can use the fedmsgs Nils
already sent in this thread.
Regards,
Jan Kaluza
Post by Adam Williamson
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
_______________________________________________
_______________________________________________
Michal Novotny
2018-03-23 06:24:58 UTC
Permalink
Post by Jan Kaluza
Post by Michal Novotny
The question here is why don't we actually implement the modularity
the simple way.
Let's suppose I would like to make nodejs-8 available in EPEL7.
The most effortless way would be to create subdirectory called 8 at
src.fedoraproject.org/rpms/nodejs
with the updated spec file and sources for the bumped major version of
the nodejs package.
What if your module contains packages coming from multiple SRPMs and they
depend on each other?
Well, my module wouldn't be coming from multiple SRPMs because my
module is just a normal package.

The problem with modularity is that the name does not really fit what
it does, which is kind of important because wrong names make
implementation much harder as you cannot rely on your common
sense.

When I think of a module, rpm ultimately comes to mind. That's kind of
a unit that makes up an operating system.

I highly doubt modularity can change it unless it invents its own package
format. So I think it would be good to call it differently even in the tooling.

So let's try to figure out a better name for modulemd.

Basically, the final product of modulemd is a repo with some built rpm
packages in it (if I saw the latest development correctly, we were squashing
multiple of those repos into a single one but we wouldn't need to do that).

For users, a repo represents something like a stream of content. So
streammd would be already a better name.

But even a better name would be coprmd because you can think about
the current "modulemd" as a copr (community project) being formally
described in a file. Product of a copr project is also a repo so it fits.

Stream name here:

https://github.com/fedora-modularity/libmodulemd/blob/master/spec.v2.yaml#L13

is basically a project name.


The content specified here:

https://github.com/fedora-modularity/libmodulemd/blob/master/spec.v2.yaml#L305

are basically definitions of SCM packages.

And then there are some fields like:

version: 20160927144203

context: c0ffee43

which doesn't really belong there imho but instead belong to the final
built repository (= copr snapshot) as a part of repodata (if anywhere).

So I think it would be good to keep talking about packages when
we talk about modules.

clime
Post by Jan Kaluza
Jan Kaluza
Post by Michal Novotny
For fedpkg to be able to upload and download sources for that
subdirectory (submodule), the attached
single-line patch is needed in python-rpkg. There are a few more
changes needed to make srpm
importing work but it should be basically almost done as well.
Koji then just needs to learn how to build DistGit repositories
containing submodules where
submodule is any subdirectory containing a spec file. When Koji
discovers all the submodules
in a repository, it can start an srpm and a consequent rpm build for
each of them.
It is a very similar approach to what find_packages from python
setuptools implements. It
recognizes subpackages based on the presence of __init__.py file. The
only difference here
is that we will be recognizing subpackages (=submodules) by the
presence of *.spec file.
Everything else should be pretty much the same otherwise. Koji builds
nodejs-6 and nodejs-8 rpms
both with el7 disttag and both of these can go through the Bodhi
update process separately.
Of course, the question is if you can actually build and run nodejs-8
in EL7 successfully but
I think that will always be a problem no matter what the implementation is.
clime
On Mon, Mar 19, 2018 at 4:43 PM, Adam Williamson
Post by Adam Williamson
Post by Stanislav Ochotnicky
Post by Fabio Valentini
So you think having to send a request to a web service instead of just
parsing a string locally with one line of code is a good trade-off for
allowing dashes?
This has been mentioned several times in this thread and I think there's
a misunderstanding around this.
So: When your tool/whatever works with modules it will have to have
module metadata available in some form.
What if I don't want to work with modules at all, just with information
about modules?
What if I just want to write a tool that parses fedmsgs about module
builds in Koji and does something that depends on module names, streams
and versions? (e.g. some sort of changelog thing)
Your tool can use good old buildsys.build.state.change fedmsg with the well
https://apps.stg.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.stg.buildsys.build.state.change
That's staging, because it's easier for me to find the module builds there
:).
Unfortunatelly, Koji does not add "type" field there, so you cannot find out
if that's module or not. But in case your tool is doing general changelog
per Koji build, it will work quite well with modules without you even
noticing you are working with modules.
In case of tools, you can even query koji using its api and pass the
name/version/release there, instead of NVR.
If you query Koji for the module build, you will actually find whole module
metadata in "extra" part of the build.
In case you are OK with using MBS fedmsgs, you can use the fedmsgs Nils
already sent in this thread.
Regards,
Jan Kaluza
Post by Adam Williamson
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
_______________________________________________
_______________________________________________
_______________________________________________
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an ema
Michal Novotny
2018-03-23 06:33:53 UTC
Permalink
Post by Michal Novotny
Post by Jan Kaluza
Post by Michal Novotny
The question here is why don't we actually implement the modularity
the simple way.
Let's suppose I would like to make nodejs-8 available in EPEL7.
The most effortless way would be to create subdirectory called 8 at
src.fedoraproject.org/rpms/nodejs
with the updated spec file and sources for the bumped major version of
the nodejs package.
What if your module contains packages coming from multiple SRPMs and they
depend on each other?
Well, my module wouldn't be coming from multiple SRPMs because my
module is just a normal package.
The problem with modularity is that the name does not really fit what
it does, which is kind of important because wrong names make
implementation much harder as you cannot rely on your common
sense.
When I think of a module, rpm ultimately comes to mind. That's kind of
a unit that makes up an operating system.
I highly doubt modularity can change it unless it invents its own package
format. So I think it would be good to call it differently even in the tooling.
So let's try to figure out a better name for modulemd.
Basically, the final product of modulemd is a repo with some built rpm
packages in it (if I saw the latest development correctly, we were squashing
multiple of those repos into a single one but we wouldn't need to do that).
For users, a repo represents something like a stream of content. So
streammd would be already a better name.
But even a better name would be coprmd because you can think about
the current "modulemd" as a copr (community project) being formally
described in a file. Product of a copr project is also a repo so it fits.
https://github.com/fedora-modularity/libmodulemd/blob/master/spec.v2.yaml#L13
is basically a project name.
https://github.com/fedora-modularity/libmodulemd/blob/master/spec.v2.yaml#L305
are basically definitions of SCM packages.
That link should have pointed here:
https://github.com/fedora-modularity/libmodulemd/blob/master/spec.v2.yaml#L233
Post by Michal Novotny
version: 20160927144203
context: c0ffee43
which doesn't really belong there imho but instead belong to the final
built repository (= copr snapshot) as a part of repodata (if anywhere).
So I think it would be good to keep talking about packages when
we talk about modules.
clime
Post by Jan Kaluza
Jan Kaluza
Post by Michal Novotny
For fedpkg to be able to upload and download sources for that
subdirectory (submodule), the attached
single-line patch is needed in python-rpkg. There are a few more
changes needed to make srpm
importing work but it should be basically almost done as well.
Koji then just needs to learn how to build DistGit repositories
containing submodules where
submodule is any subdirectory containing a spec file. When Koji
discovers all the submodules
in a repository, it can start an srpm and a consequent rpm build for
each of them.
It is a very similar approach to what find_packages from python
setuptools implements. It
recognizes subpackages based on the presence of __init__.py file. The
only difference here
is that we will be recognizing subpackages (=submodules) by the
presence of *.spec file.
Everything else should be pretty much the same otherwise. Koji builds
nodejs-6 and nodejs-8 rpms
both with el7 disttag and both of these can go through the Bodhi
update process separately.
Of course, the question is if you can actually build and run nodejs-8
in EL7 successfully but
I think that will always be a problem no matter what the implementation is.
clime
On Mon, Mar 19, 2018 at 4:43 PM, Adam Williamson
Post by Adam Williamson
Post by Stanislav Ochotnicky
Post by Fabio Valentini
So you think having to send a request to a web service instead of just
parsing a string locally with one line of code is a good trade-off for
allowing dashes?
This has been mentioned several times in this thread and I think there's
a misunderstanding around this.
So: When your tool/whatever works with modules it will have to have
module metadata available in some form.
What if I don't want to work with modules at all, just with information
about modules?
What if I just want to write a tool that parses fedmsgs about module
builds in Koji and does something that depends on module names, streams
and versions? (e.g. some sort of changelog thing)
Your tool can use good old buildsys.build.state.change fedmsg with the well
https://apps.stg.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.stg.buildsys.build.state.change
That's staging, because it's easier for me to find the module builds there
:).
Unfortunatelly, Koji does not add "type" field there, so you cannot find out
if that's module or not. But in case your tool is doing general changelog
per Koji build, it will work quite well with modules without you even
noticing you are working with modules.
In case of tools, you can even query koji using its api and pass the
name/version/release there, instead of NVR.
If you query Koji for the module build, you will actually find whole module
metadata in "extra" part of the build.
In case you are OK with using MBS fedmsgs, you can use the fedmsgs Nils
already sent in this thread.
Regards,
Jan Kaluza
Post by Adam Williamson
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
_______________________________________________
_______________________________________________
_______________________________________________
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedorapr

Kamil Paral
2018-03-20 14:18:40 UTC
Permalink
Post by Jan Kaluza
Your tool can use good old buildsys.build.state.change fedmsg with the
https://apps.stg.fedoraproject.org/datagrepper/
raw?topic=org.fedoraproject.stg.buildsys.build.state.change
Initially I was very concerned about the idea of having to query Koji for
each module to get NSVC, because in Taskotron we run ten thousand tasks per
day and every unnecessary network request means slower execution, higher
risk of failure and more logs to debug. We try to minimize the number of
network requests as much as we can. However, if the structured metadata
is/will be available in fedmsgs, I think I don't have a use case at hand
that I could use to object to this. I still think it's a bad idea, though,
and will bite us in the long run.
Post by Jan Kaluza
That's staging, because it's easier for me to find the module builds there
:).
Unfortunatelly, Koji does not add "type" field there, so you cannot find
out if that's module or not.
Is that something that will get resolved soon? Because it seems like a big
blocker to any automated processing. If we want to run some tasks just on
rpms and some tasks just on modules, how do we distinguish that?
Jan Kaluza
2018-03-20 14:40:50 UTC
Permalink
Post by Kamil Paral
Post by Jan Kaluza
Your tool can use good old buildsys.build.state.change fedmsg with the
https://apps.stg.fedoraproject.org/datagrepper/raw?topic=
org.fedoraproject.stg.buildsys.build.state.change
Initially I was very concerned about the idea of having to query Koji for
each module to get NSVC, because in Taskotron we run ten thousand tasks per
day and every unnecessary network request means slower execution, higher
risk of failure and more logs to debug. We try to minimize the number of
network requests as much as we can. However, if the structured metadata
is/will be available in fedmsgs, I think I don't have a use case at hand
that I could use to object to this. I still think it's a bad idea, though,
and will bite us in the long run.
Post by Jan Kaluza
That's staging, because it's easier for me to find the module builds
there :).
Unfortunatelly, Koji does not add "type" field there, so you cannot find
out if that's module or not.
Is that something that will get resolved soon? Because it seems like a big
blocker to any automated processing. If we want to run some tasks just on
rpms and some tasks just on modules, how do we distinguish that?
You can query the Koji for build id and check the type of a build.

I don't think that feature request is even reported to Koji upstream,
because nobody needed that and I found that yesterday.

It is however not new situation. There is the same "issue" for "images". If
you want to run some tasks just for RPMs and not for images, you have to do
the filtering like this anyway. Therefore I don't really consider this as
real blocker which needs fix urgently.
Post by Kamil Paral
_______________________________________________
Adam Williamson
2018-03-19 15:34:23 UTC
Permalink
Post by Petr Å abata
This, AFAIK, only affects what you see in some parts of the
build and update system (and on the bus).
That's a very important parenthesis, as we as a project do a lot of
work with fedmsg messages.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedor
Owen Taylor
2018-03-16 20:57:53 UTC
Permalink
Post by Dennis Gregorovic
The challenge here is that those module builds are squeezed into a N-V-R
format when imported into Koji. The N-V-R format is used for all build
types in Koji and it works well for its purpose, but it's also not
realistic to expect that all content types are going to natively use N-V-R
outside of Koji. My suggestion is that we consider the N-V-R format of
modules to be a representation internal to koji and that N:S:V:C is the
format used to represent modules to users. If that means tweaking the Koji
UI we can look into that.
There's also a terminology problem that gets less confusing with the :
separator. When compared to

Name-VERSION-Release.DistTag

it would be fairly confusing to have:

Name-Stream-VERSION.Context

while:

Name:Stream:VERSION:Context

is slightly less confusing. of course, it might have been better to call
'Version' in a module version Build or something, but I think that ship has
sailed.

In the end, modules are not packages, and it's necessarily required or good
if they are named in the same format. I'd also like to see Koji's
representation of a module version stay inside Koji as much as possible.

Owen
Adam Williamson
2018-03-16 22:19:42 UTC
Permalink
Post by Dennis Gregorovic
modules are not RPMs. I would not expect them to necessarily use the same
format as RPMs. If we take koji out of the equation,
That's an extremely big and invalid 'if', though.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscri
Dennis Gilmore
2018-03-17 03:35:35 UTC
Permalink
Post by Adam Williamson
Post by Dennis Gregorovic
modules are not RPMs. I would not expect them to necessarily use the same
format as RPMs. If we take koji out of the equation,
That's an extremely big and invalid 'if', though.
koji is mirroring rpm's behaviour, the if would have to be if you took
rpm out of the equation.

Dennis
Kevin Kofler
2018-03-16 04:01:25 UTC
Permalink
Post by Randy Barlow
It sounds like this breaks a great deal of tooling. Can we reconsider
switching away from - separators in modules? Is allowing streams to have
-'s in them important enough to break so many tools?
+1, I think allowing this ambiguity is a very bad idea.

Kevin Kofler
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lis
Adam Williamson
2018-03-15 16:26:40 UTC
Permalink
Post by Patrick Uiterwijk
One of the interesting things they wanted to also allow: dashes in streams.
As a consequence, when you get an N-S-V.C as modules are represented
in Koji builds, doing a .rsplit('-', 2) will not give you Name,
Stream, Version.Context per se.
You could totally have a module called
nodejs-my-stream-5-20170314.abcd, with name=nodejs,
stream=my-stream-5, version=20170314, context=abcd.
There is no way for you to independently figure out what the NSVC
components are, you will need to ask Koji, and use its name, version
and release fields (with name=name, version=stream,
release=version(.context)).
...
Post by Patrick Uiterwijk
I hope that this is useful information for anyone else finding
themselves having to parse NVRs/NSVs.
1. Yes, it helps, thanks.
2. This is goddamn terrible and whoever is responsible should be
ashamed.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to
Loading...