Lifecycle objective - problems, solutions, and proposal

[Paul Frields]

I worked with FPL Matthew Miller and our engineering manager Jim
Perrin, among others, to define the various problems we want to solve
in diversifying the Fedora lifecycle. We're seeking review and
feedback from community members. The most salient feedback will be
from those involved in the efforts we describe, but we welcome all
constructive feedback.

Here's the summary from the page, which proposes we pause the release
after F30 for these efforts:

* * *
Fedora’s singular lifecycle has been in place for almost a decade and
a half. During that time, technology users have changed how they
consume platform and applications. Fedora needs to be more toward the
forefront of these changes. But more importantly, Fedora needs to be
more hospitable to community management of lifecycle.

Currently Fedora can’t scale for more community ownership of the
things we release: (1) Only a few people can build and push out
releases; and (2) we manage releases largely based on that staffing.
The Fedora community should be able to run releases of content
themselves, using tools that work well, with only minimal oversight,
and determine their own schedule for doing so.

This implies a great deal of both redesign and reworking of tools and
processes. To unblock the community, several things need to happen. We
need a faster, more scalable compose to enable CI/CD operations; we
need to automate more testing and quality measures; and we need to
update our delivery tools and processes. We also need to track and
coordinate this work across teams, since it involves collaboration
among Fedora infrastructure, QA, applications, release engineering,
CentOS CI, maintainers, and more.

We should skip the F31 release cycle and leave F30 in place longer in
order to focus on improving the tooling and testing changes. These
tooling changes will improve the overall reliability of Fedora, and
will decrease the manual effort and complexities involved in producing
the distribution artifacts. Although we’ve done this before to make
“editions” happen, the intent is to track this multi-team effort more
actively so we can (1) use the time as well as possible, and (2) give
the work maximum transparency.
* * *

The full page is here, with a set of problems, solutions, and actions
proposed. I invite you to take time to read it in detail:
https://fedoraproject.org/wiki/Objectives/Lifecycle/Problem_statements

I also attached that page to the main Lifecycle objective page here,
which was previously approved by the Council:
https://fedoraproject.org/wiki/Objectives/Lifecycle

Rather than try to spin this one email into a thread of doom that
people will give up on reading, I encourage those with feedback to
open a thread for any particular topic. That way the community
discussion should be more useful for all.

I'll collect input and use it both for responses and to help tweak
plans for (hopefully) optimal results. I'm on vacation next week, for
which timing I apologize. But I'll try to look at mail here from time
to time and when I return.

[Matthew Miller]

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
proposals:

* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)
* really actually for real gated Rawhide
* better CI pipeline tests for everything
* define a base platform -- Red Hat wants to focus resources here
* better tooling for non-base deliverables
* better metrics for everything


Please read https://fedoraproject.org/wiki/Objectives/Lifecycle/Problem_statements
and comment in this thread.

--
Matthew Miller
<***@fedoraproject.org>
Fedora Project Leader
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archiv

[Brian (bex) Exelbierd]

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)
* really actually for real gated Rawhide
* better CI pipeline tests for everything
* define a base platform -- Red Hat wants to focus resources here
* better tooling for non-base deliverables
* better metrics for everything

I am +100 to these changes.

The idea that we can essentially not release for a year to do this
raises the interesting point that perhaps our release cadence is not
as rigid as we think it may be. I don't think we are going to skip
Gnome and other major updates, are we? If not, then this also runs
into the conversation about longer lifecycles and whether we need
releases, imho.

regards,

bex

Post by Matthew Miller
Please read https://fedoraproject.org/wiki/Objectives/Lifecycle/Problem_statements
and comment in this thread.
--
Matthew Miller
Fedora Project Leader
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

--
Brian (bex) Exelbierd | ***@redhat.com | ***@pobox.com
Fedora Community Action & Impact Coordinator
@bexelbie | http://www.winglemeyer.org
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lis

[Peter Robinson]

On Mon, Nov 26, 2018 at 4:23 PM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)
* really actually for real gated Rawhide
* better CI pipeline tests for everything
* define a base platform -- Red Hat wants to focus resources here
* better tooling for non-base deliverables
* better metrics for everything

I am +100 to these changes.
The idea that we can essentially not release for a year to do this
raises the interesting point that perhaps our release cadence is not
as rigid as we think it may be. I don't think we are going to skip
Gnome and other major updates, are we? If not, then this also runs
into the conversation about longer lifecycles and whether we need
releases, imho.

I like some of the ideas here but I feel we need to have an extremely
well defined set of problems to be solved else we'll delay the
release, achieve very little, and either never release again or not
have anything achieved except a waste of time.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedora

[Kevin Fenzi]

Post by Brian (bex) Exelbierd

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)
* really actually for real gated Rawhide
* better CI pipeline tests for everything
* define a base platform -- Red Hat wants to focus resources here
* better tooling for non-base deliverables
* better metrics for everything

I am +100 to these changes.
The idea that we can essentially not release for a year to do this
raises the interesting point that perhaps our release cadence is not
as rigid as we think it may be. I don't think we are going to skip
Gnome and other major updates, are we? If not, then this also runs
into the conversation about longer lifecycles and whether we need
releases, imho.

Indeed.

If we push as updates things we normally wait for releases for, would
that disrupt our users?

Is there a cutoff of things we would push in updates? For example, we
usually do mass rebuilds of the entire collection of packages when new
gcc/etc are ready, would we do that as a update in a stable release?

We could solve the rolling changes issue by requiring all major changes
to be modular and not enable the new module (ie, gnome 3.32 releases in
f30, in 6 mmonths 3.34 comes out, we enable it as a new module, users
can choose to switch to if they want, or stay on 3.32 if they ignore it
or don't want to right then). That won't help for tool changes tho.

kevin

[Igor Gnatenko]

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning

That would be good, however I don't know how it is related to
lifecycle of distro.

P.S. would be great if someone finishes packaging of it (I did a lot
of work here, but not everything).

Post by Matthew Miller
* fix the compose speed (target: one hour!)

Do we have analysis of what is taking how much time?

Post by Matthew Miller
* really actually for real gated Rawhide

Is the "creating side tag for chain builds or by automated requests"
is planned here?
When I deal with rust packages, I often need to build some update
which will break other packages and I need to fix them, but if
something will prevent going that build in buildroot.

Post by Matthew Miller
* better CI pipeline tests for everything

Does it mean that finally someone will make it usable? Do we have
definition of "better"? I've tried to use CI for all rust-* packages,
but it is basically unusable.

* No way to re-trigger tests
* Koji repo is broken even is enabled
(https://pagure.io/fedora-ci/general/issue/14, no single comment for >
0.5 month)
* Doesn't re-trigger after dependency changes
* No way to tell that set of tests should be run on some file pattern
(copy-cating same tests folder to 480 packages is not much fun)

Post by Matthew Miller
* define a base platform -- Red Hat wants to focus resources here

Can you elaborate?

Post by Matthew Miller
* better tooling for non-base deliverables

Are Ursa Major and Freshmaker included in here?

Post by Matthew Miller
* better metrics for everything
Please read https://fedoraproject.org/wiki/Objectives/Lifecycle/Problem_statements
and comment in this thread.

I would appreciate having this thread on discourse because since
introduction it is my main place where I try to have such
conversations.

Post by Matthew Miller
--
Matthew Miller
Fedora Project Leader
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/dev

[Randy Barlow]

Post by Igor Gnatenko

Post by Matthew Miller
* really actually for real gated Rawhide

Is the "creating side tag for chain builds or by automated requests"
is planned here?
When I deal with rust packages, I often need to build some update
which will break other packages and I need to fix them, but if
something will prevent going that build in buildroot.

Hey Igor!

Indeed, I do plan to have Bodhi manage side tags in order to accomplish
this. There's a Bodhi kanban board that covers the various items
planned to be part of this:

https://github.com/fedora-infra/bodhi/projects/3

You can see that a few of the cards in there relate to side tags. My
current thinking is that you will be able to create a side tag update,
which will work slightly different than a normal update in a few ways:

* You create the side tag update before you make any builds, instead of
after.
* The side tag update makes, well, a side tag in Koji for you.
* You go build all the builds you need in your side tag.
* When you are done, you tell Bodhi you are ready to go to testing, and
it will work like a normal update from there on.

Does that sound useful to you?

[Igor Gnatenko]

Post by Randy Barlow

Post by Igor Gnatenko

Post by Matthew Miller
* really actually for real gated Rawhide

Is the "creating side tag for chain builds or by automated requests"
is planned here?
When I deal with rust packages, I often need to build some update
which will break other packages and I need to fix them, but if
something will prevent going that build in buildroot.

Hey Igor!

Hey Randy,

Indeed, I do plan to have Bodhi manage side tags in order to accomplish

Post by Randy Barlow
this. There's a Bodhi kanban board that covers the various items
https://github.com/fedora-infra/bodhi/projects/3
You can see that a few of the cards in there relate to side tags. My
current thinking is that you will be able to create a side tag update,
* You create the side tag update before you make any builds, instead of
after.
* The side tag update makes, well, a side tag in Koji for you.
* You go build all the builds you need in your side tag.
* When you are done, you tell Bodhi you are ready to go to testing, and
it will work like a normal update from there on.
Does that sound useful to you?

It definitely does, just wanted to ensure if it is one of things which is
supposed to be worked on if we skip release.

_______________________________________________

Post by Randy Barlow
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

[Vít Ondruch]

Post by Randy Barlow

Post by Igor Gnatenko

Post by Matthew Miller
* really actually for real gated Rawhide

Is the "creating side tag for chain builds or by automated requests"
is planned here?
When I deal with rust packages, I often need to build some update
which will break other packages and I need to fix them, but if
something will prevent going that build in buildroot.

Hey Igor!
Indeed, I do plan to have Bodhi manage side tags in order to accomplish
this.

Does that mean that Bodhi will be finally enabled for Rawhide or not?
Because otherwise I can't see how Bodhi is related here.


Vít

Post by Randy Barlow
There's a Bodhi kanban board that covers the various items
https://github.com/fedora-infra/bodhi/projects/3
You can see that a few of the cards in there relate to side tags. My
current thinking is that you will be able to create a side tag update,
* You create the side tag update before you make any builds, instead of
after.
* The side tag update makes, well, a side tag in Koji for you.
* You go build all the builds you need in your side tag.
* When you are done, you tell Bodhi you are ready to go to testing, and
it will work like a normal update from there on.
Does that sound useful to you?
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

[Randy Barlow]

Post by Vít Ondruch
Does that mean that Bodhi will be finally enabled for Rawhide or not?
Because otherwise I can't see how Bodhi is related here.

Indeed, the plan is to make Bodhi manage Rawhide as well.

[m***@gnome.org]

On Mon, Nov 26, 2018 at 10:03 AM, Matthew Miller

Post by Matthew Miller
I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level

Can we use this as an opportunity to work out how to use the blocker
bugs process for a megaupdate? Our reputation for providing an
up-to-date desktop took a hit last time we did this, so we'll want to
provide a GNOME 3.34 megaupdate in October 2019 to compensate for the
missing Fedora release there. We'll need a monthslong testing process
for this to go smoothly mid-release.

Michael
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedor

[Peter Robinson]

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)

Can I have a unicorn? Everyone wants this bug absolutely no one has
done analysis. The fact of the matter is that the compose has a LOT of
I/O and I/O is slow and the cost to upgrade the infrastructure to get
high through put I/O is expensive and no one has ever offered the
budget to resolve that problem. The fact of the matter with this is
that we now produce a lot of release artifacts and that takes time,
some of it can probably be run more in parallel, and possibly some
things run at different times but without a combination of investment
in infrastructure, and maybe even hacking and slashing of components
this is not simply a check box.

For example the IoT compose takes around an hour but ATM we currently
produce 5 artifacts, mostly in parallel, and don't do things like
newRepo for the base rpms because we consume that from the main
compose.

Post by Matthew Miller
* really actually for real gated Rawhide
* better CI pipeline tests for everything
* define a base platform -- Red Hat wants to focus resources here
* better tooling for non-base deliverables
* better metrics for everything

They are very nice top level ideas, but each of those components need
to have in depth analysis and not just the hand wavy overview in the
link below.

Post by Matthew Miller
Please read https://fedoraproject.org/wiki/Objectives/Lifecycle/Problem_statements
and comment in this thread.
--
Matthew Miller
Fedora Project Leader
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists

[Adam Williamson]

Post by Peter Robinson

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)

Can I have a unicorn? Everyone wants this bug absolutely no one has
done analysis.

I just don't believe this is true. I'm pretty sure Dennis and Kevin
have both looked into it before.

Post by Peter Robinson
The fact of the matter is that the compose has a LOT of
I/O and I/O is slow and the cost to upgrade the infrastructure to get
high through put I/O is expensive and no one has ever offered the
budget to resolve that problem. The fact of the matter with this is
that we now produce a lot of release artifacts and that takes time,
some of it can probably be run more in parallel, and possibly some
things run at different times but without a combination of investment
in infrastructure, and maybe even hacking and slashing of components
this is not simply a check box.

That's why the proposal is to skip an entire release to give us time to
work on it. If it was easy, that wouldn't be necessary.

There are of course at least *two* possible solutions to "there's lots
of I/O". One is "throw money at making the I/O faster", the other is
"figure out how to do less I/O".

One hour is a *target*, not a *requirement*. I don't think anyone's
going to consider the time wasted if we wind up getting it down to two
hours instead of one.

(Though personally I'd like ten minutes. And two unicorns.)
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/arc

[Peter Robinson]

On Mon, Nov 26, 2018 at 6:53 PM Adam Williamson

Post by Adam Williamson

Post by Peter Robinson

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)

Can I have a unicorn? Everyone wants this bug absolutely no one has
done analysis.

I just don't believe this is true. I'm pretty sure Dennis and Kevin
have both looked into it before.

Yes, I mean recently, I was involved in the last time we attempted
this, and there was a number of things identified but quite a bit has
changed since that last happened.

Post by Adam Williamson

Post by Peter Robinson
The fact of the matter is that the compose has a LOT of
I/O and I/O is slow and the cost to upgrade the infrastructure to get
high through put I/O is expensive and no one has ever offered the
budget to resolve that problem. The fact of the matter with this is
that we now produce a lot of release artifacts and that takes time,
some of it can probably be run more in parallel, and possibly some
things run at different times but without a combination of investment
in infrastructure, and maybe even hacking and slashing of components
this is not simply a check box.

That's why the proposal is to skip an entire release to give us time to
work on it. If it was easy, that wouldn't be necessary.
There are of course at least *two* possible solutions to "there's lots
of I/O". One is "throw money at making the I/O faster", the other is
"figure out how to do less I/O".
One hour is a *target*, not a *requirement*. I don't think anyone's
going to consider the time wasted if we wind up getting it down to two
hours instead of one.
(Though personally I'd like ten minutes. And two unicorns.)
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedoraproj

[Kevin Fenzi]

Post by Peter Robinson
On Mon, Nov 26, 2018 at 6:53 PM Adam Williamson

Post by Adam Williamson

Post by Peter Robinson

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)

Can I have a unicorn? Everyone wants this bug absolutely no one has
done analysis.

I just don't believe this is true. I'm pretty sure Dennis and Kevin
have both looked into it before.

Yes, I mean recently, I was involved in the last time we attempted
this, and there was a number of things identified but quite a bit has
changed since that last happened.

Yeah, I think our goal should be 1 minute... just as realistic as one
hour. ;)

I have not looked into things recently. However:

* The mock fsync change (https://pagure.io/releng/issue/7909) may help
some.

* We have plans to redo the setup on the s390x builders, which should
result in them being much faster. That should help.

* I know pungi maintaienrs have been doing some work to make things
faster (even in the last release out this morning).

All that said, I am not sure there's going to be a way to get it down to
an hour. I think getting it down to 3-4hours may be very helpful tho and
might be possible.

kevin

[Adam Williamson]

Post by Kevin Fenzi

Post by Peter Robinson
On Mon, Nov 26, 2018 at 6:53 PM Adam Williamson

Post by Adam Williamson

Post by Peter Robinson

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)

Can I have a unicorn? Everyone wants this bug absolutely no one has
done analysis.

I just don't believe this is true. I'm pretty sure Dennis and Kevin
have both looked into it before.

Yes, I mean recently, I was involved in the last time we attempted
this, and there was a number of things identified but quite a bit has
changed since that last happened.

Yeah, I think our goal should be 1 minute... just as realistic as one
hour. ;)
* The mock fsync change (https://pagure.io/releng/issue/7909) may help
some.
* We have plans to redo the setup on the s390x builders, which should
result in them being much faster. That should help.
* I know pungi maintaienrs have been doing some work to make things
faster (even in the last release out this morning).
All that said, I am not sure there's going to be a way to get it down to
an hour. I think getting it down to 3-4hours may be very helpful tho and
might be possible.

While we're on the topic of unicorns, if I were the person actually in
charge of this and had the freedom to do it how I wanted, what I'd
really want to do is create a much more *modular* compose process,
where inputs and outputs were defined and associated with each other,
and you could easily define subsets to be produced at different times
for different reasons. Not a compose, but a Compose Construction Kit. A
big chunk of the length of "the compose" is associated with the fact
that we define "the compose" as a single thing which produces (last
time I counted) 70+ deliverables. If we had an easy way to say 'ok,
right now we need a compose which produces A, B, C and D from the
minimum necessary sets of inputs', that would be a massive improvement.

This hasn't been true for a while (we actually have something like half
a dozen or more different "composes", right now), but the system is
still less flexible and it's still more difficult to configure
different composes than it really ought to be.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/lis

[Peter Robinson]

On Mon, Nov 26, 2018 at 7:42 PM Adam Williamson

Post by Adam Williamson

Post by Kevin Fenzi

Post by Peter Robinson
On Mon, Nov 26, 2018 at 6:53 PM Adam Williamson

Post by Adam Williamson

Post by Peter Robinson

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)

Can I have a unicorn? Everyone wants this bug absolutely no one has
done analysis.

I just don't believe this is true. I'm pretty sure Dennis and Kevin
have both looked into it before.

Yes, I mean recently, I was involved in the last time we attempted
this, and there was a number of things identified but quite a bit has
changed since that last happened.

Yeah, I think our goal should be 1 minute... just as realistic as one
hour. ;)
* The mock fsync change (https://pagure.io/releng/issue/7909) may help
some.
* We have plans to redo the setup on the s390x builders, which should
result in them being much faster. That should help.
* I know pungi maintaienrs have been doing some work to make things
faster (even in the last release out this morning).
All that said, I am not sure there's going to be a way to get it down to
an hour. I think getting it down to 3-4hours may be very helpful tho and
might be possible.

While we're on the topic of unicorns, if I were the person actually in
charge of this and had the freedom to do it how I wanted, what I'd
really want to do is create a much more *modular* compose process,
where inputs and outputs were defined and associated with each other,
and you could easily define subsets to be produced at different times
for different reasons. Not a compose, but a Compose Construction Kit. A
big chunk of the length of "the compose" is associated with the fact
that we define "the compose" as a single thing which produces (last
time I counted) 70+ deliverables. If we had an easy way to say 'ok,
right now we need a compose which produces A, B, C and D from the
minimum necessary sets of inputs', that would be a massive improvement.
This hasn't been true for a while (we actually have something like half
a dozen or more different "composes", right now), but the system is
still less flexible and it's still more difficult to configure
different composes than it really ought to be.

Yes, that was one of the ideas I had. Basically do the "newRepo" part
of the compose that generates the new set of tagged in and signed rpms
and base network isntallers 3-4 times a day and then run all the other
components separately (live/arm images, CoreOS, IoT, cloud images) on
their own schedule in separate components consuming the updates. This
is basically what some of the bits do now as you'd indicated. It then
allows focus to be put on each component to speed each piece up
independently rather than as a whole.

Peter
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorapro

[Mohan Boddu]

Post by Paul Frields

Post by Kevin Fenzi

Post by Peter Robinson
On Mon, Nov 26, 2018 at 6:53 PM Adam Williamson

Post by Adam Williamson

On Mon, Nov 26, 2018 at 4:05 PM Matthew Miller <

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the

release

Post by Kevin Fenzi

Post by Peter Robinson

Post by Adam Williamson

Post by Matthew Miller
I know it was a big time-off holiday week in the US, but I

expected a little

Post by Kevin Fenzi

Post by Peter Robinson

Post by Adam Williamson

Post by Matthew Miller
more interest in this post. Perhaps it seemed like too much text

to digest

Post by Kevin Fenzi

Post by Peter Robinson

Post by Adam Williamson

Post by Matthew Miller
along with turkey and stuffing. :) I'm highlighting it with a

subject

Post by Kevin Fenzi

Post by Peter Robinson

Post by Adam Williamson

Post by Matthew Miller
reflecting the big, direct impact, and here's some other

top-level

Post by Kevin Fenzi

Post by Peter Robinson

Post by Adam Williamson

Post by Matthew Miller
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)

Can I have a unicorn? Everyone wants this bug absolutely no one has
done analysis.

I just don't believe this is true. I'm pretty sure Dennis and Kevin
have both looked into it before.

Yes, I mean recently, I was involved in the last time we attempted
this, and there was a number of things identified but quite a bit has
changed since that last happened.

Yeah, I think our goal should be 1 minute... just as realistic as one
hour. ;)
* The mock fsync change (https://pagure.io/releng/issue/7909) may help
some.
* We have plans to redo the setup on the s390x builders, which should
result in them being much faster. That should help.
* I know pungi maintaienrs have been doing some work to make things
faster (even in the last release out this morning).
All that said, I am not sure there's going to be a way to get it down to
an hour. I think getting it down to 3-4hours may be very helpful tho and
might be possible.

While we're on the topic of unicorns, if I were the person actually in
charge of this and had the freedom to do it how I wanted, what I'd
really want to do is create a much more *modular* compose process,
where inputs and outputs were defined and associated with each other,
and you could easily define subsets to be produced at different times
for different reasons. Not a compose, but a Compose Construction Kit. A
big chunk of the length of "the compose" is associated with the fact
that we define "the compose" as a single thing which produces (last
time I counted) 70+ deliverables. If we had an easy way to say 'ok,
right now we need a compose which produces A, B, C and D from the
minimum necessary sets of inputs', that would be a massive improvement.
This hasn't been true for a while (we actually have something like half
a dozen or more different "composes", right now), but the system is
still less flexible and it's still more difficult to configure
different composes than it really ought to be.

This would be great.

I want to get to a state where different WG's can run their own composes
by calling a service.

They might actually release on their own schedules as well. (Not all, but
something like Labs or Spins) .

Post by Paul Frields
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

[Paul Frields]

On Mon, Nov 26, 2018 at 2:42 PM Adam Williamson

Post by Adam Williamson

Post by Kevin Fenzi
Yeah, I think our goal should be 1 minute... just as realistic as one
hour. ;)
* The mock fsync change (https://pagure.io/releng/issue/7909) may help
some.
* We have plans to redo the setup on the s390x builders, which should
result in them being much faster. That should help.
* I know pungi maintaienrs have been doing some work to make things
faster (even in the last release out this morning).
All that said, I am not sure there's going to be a way to get it down to
an hour. I think getting it down to 3-4hours may be very helpful tho and
might be possible.

While we're on the topic of unicorns, if I were the person actually in
charge of this and had the freedom to do it how I wanted, what I'd
really want to do is create a much more *modular* compose process,
where inputs and outputs were defined and associated with each other,
and you could easily define subsets to be produced at different times
for different reasons. Not a compose, but a Compose Construction Kit. A
big chunk of the length of "the compose" is associated with the fact
that we define "the compose" as a single thing which produces (last
time I counted) 70+ deliverables. If we had an easy way to say 'ok,
right now we need a compose which produces A, B, C and D from the
minimum necessary sets of inputs', that would be a massive improvement.
This hasn't been true for a while (we actually have something like half
a dozen or more different "composes", right now), but the system is
still less flexible and it's still more difficult to configure
different composes than it really ought to be.

Adam puts his finger on an important aspect of the work, which is to
"decompose the compose" (sorry). We need the ability to produce enough
A, B, C and D (abusing his example) to do integration tests for
specific inputs, such that maintainers get feedback in a reasonable
timeframe on their builds. With that, gating Rawhide can be an actual
thing.

--
Paul
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.or

[Kevin Fenzi]

Post by Paul Frields
On Mon, Nov 26, 2018 at 2:42 PM Adam Williamson

Post by Adam Williamson
While we're on the topic of unicorns, if I were the person actually in
charge of this and had the freedom to do it how I wanted, what I'd
really want to do is create a much more *modular* compose process,
where inputs and outputs were defined and associated with each other,
and you could easily define subsets to be produced at different times
for different reasons. Not a compose, but a Compose Construction Kit. A
big chunk of the length of "the compose" is associated with the fact
that we define "the compose" as a single thing which produces (last
time I counted) 70+ deliverables. If we had an easy way to say 'ok,
right now we need a compose which produces A, B, C and D from the
minimum necessary sets of inputs', that would be a massive improvement.
This hasn't been true for a while (we actually have something like half
a dozen or more different "composes", right now), but the system is
still less flexible and it's still more difficult to configure
different composes than it really ought to be.

Agreed, that would be nice. The one issue I see off hand is that koji
tags are kind of expensive so we can't just tag everything the way we
may want to, but perhaps we can come up with some clever workflow for that.

Post by Paul Frields
Adam puts his finger on an important aspect of the work, which is to
"decompose the compose" (sorry). We need the ability to produce enough
A, B, C and D (abusing his example) to do integration tests for
specific inputs, such that maintainers get feedback in a reasonable
timeframe on their builds.

Yeah, thats long been a desire of releng/qa... weeding out the obviously
broken would be very nice.

Post by Paul Frields
With that, gating Rawhide can be an actual
thing.

Indeed.

kevin

[John Florian]

Post by Kevin Fenzi
The one issue I see off hand is that koji
tags are kind of expensive so we can't just tag everything the way we
may want

Can you please elaborate a bit on this?  What makes them expensive?
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedo

[Neal Gompa]

Post by Kevin Fenzi
The one issue I see off hand is that koji
tags are kind of expensive so we can't just tag everything the way we
may want

Can you please elaborate a bit on this? What makes them expensive?

Koji tags are representations of full repositories. They work more or
less the same way that Copr projects do, it's just a lot less obvious
because of the way Koji represents them in the various UIs.



--
真実はいつも一つ！/ Always, there's only one truth!
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/arch

[Matthew Miller]

Post by Adam Williamson
While we're on the topic of unicorns, if I were the person actually in
charge of this and had the freedom to do it how I wanted, what I'd
really want to do is create a much more *modular* compose process,
where inputs and outputs were defined and associated with each other,
and you could easily define subsets to be produced at different times
for different reasons. Not a compose, but a Compose Construction Kit. A
big chunk of the length of "the compose" is associated with the fact
that we define "the compose" as a single thing which produces (last
time I counted) 70+ deliverables. If we had an easy way to say 'ok,
right now we need a compose which produces A, B, C and D from the
minimum necessary sets of inputs', that would be a massive improvement.

+1000


--
Matthew Miller
<***@fedoraproject.org>
Fedora Project Leader
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/

[Mohan Boddu]

Post by Matthew Miller

Post by Peter Robinson
On Mon, Nov 26, 2018 at 6:53 PM Adam Williamson

Post by Adam Williamson

On Mon, Nov 26, 2018 at 4:05 PM Matthew Miller <

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a

little

Post by Peter Robinson

Post by Adam Williamson

Post by Matthew Miller
more interest in this post. Perhaps it seemed like too much text to

digest

Post by Peter Robinson

Post by Adam Williamson

Post by Matthew Miller
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)

Can I have a unicorn? Everyone wants this bug absolutely no one has
done analysis.

I just don't believe this is true. I'm pretty sure Dennis and Kevin
have both looked into it before.

Yes, I mean recently, I was involved in the last time we attempted
this, and there was a number of things identified but quite a bit has
changed since that last happened.

Yeah, I think our goal should be 1 minute... just as realistic as one
hour. ;)
* The mock fsync change (https://pagure.io/releng/issue/7909) may help
some.
* We have plans to redo the setup on the s390x builders, which should
result in them being much faster. That should help.
* I know pungi maintaienrs have been doing some work to make things
faster (even in the last release out this morning).
All that said, I am not sure there's going to be a way to get it down to
an hour. I think getting it down to 3-4hours may be very helpful tho and
might be possible.

I totally agree, also as Peter mentioned, IOT composes are just taking 1
hour as its
just consuming the repo rather than generating newRepo as the normal
nightly composes does. If we can generate the newRepo on the fly whenever
a package gets build and just use the repo to generate the artifacts either
individually (splitting the compose) or parallely (as we are doing right
now mostly)
that would really help.

Post by Matthew Miller
kevin
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

[Peter Robinson]

Post by Kevin Fenzi

Post by Peter Robinson
On Mon, Nov 26, 2018 at 6:53 PM Adam Williamson

Post by Adam Williamson

Post by Peter Robinson

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)

Can I have a unicorn? Everyone wants this bug absolutely no one has
done analysis.

I just don't believe this is true. I'm pretty sure Dennis and Kevin
have both looked into it before.

Yes, I mean recently, I was involved in the last time we attempted
this, and there was a number of things identified but quite a bit has
changed since that last happened.

Yeah, I think our goal should be 1 minute... just as realistic as one
hour. ;)
* The mock fsync change (https://pagure.io/releng/issue/7909) may help
some.
* We have plans to redo the setup on the s390x builders, which should
result in them being much faster. That should help.
* I know pungi maintaienrs have been doing some work to make things
faster (even in the last release out this morning).
All that said, I am not sure there's going to be a way to get it down to
an hour. I think getting it down to 3-4hours may be very helpful tho and
might be possible.

I totally agree, also as Peter mentioned, IOT composes are just taking 1 hour as its
just consuming the repo rather than generating newRepo as the normal
nightly composes does. If we can generate the newRepo on the fly whenever
a package gets build and just use the repo to generate the artifacts either
individually (splitting the compose) or parallely (as we are doing right now mostly)
that would really help.

Unfortunately the newRepo/installer bit is likely the most I/O
intensive, but I think it we split it out into it's own process, see
mail I just sent, it would be a good start, and then also a good
separate standalone focus for optimizations.

Peter
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/lis

[Paul Frields]

Post by Peter Robinson
On Mon, Nov 26, 2018 at 6:53 PM Adam Williamson

Post by Adam Williamson

Post by Peter Robinson

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)

Can I have a unicorn? Everyone wants this bug absolutely no one has
done analysis.

I just don't believe this is true. I'm pretty sure Dennis and Kevin
have both looked into it before.

Yes, I mean recently, I was involved in the last time we attempted
this, and there was a number of things identified but quite a bit has
changed since that last happened.

One of the large factors -- which I think was noted in the page -- was
RPM scriptlets. Stephen Gallagher met with WIll Woods and James Antill
just recently to talk about this. The time savings was substantial.
Combining that with reducing the size of "speculative composes" that
could be used for CI and other pre-ship purposes could get us to where
we want to be. Stephen's carrying a heavy duty project at the moment
but when that's done in a few weeks, this will be getting more
attention.

Post by Peter Robinson

Post by Adam Williamson

Post by Peter Robinson
The fact of the matter is that the compose has a LOT of
I/O and I/O is slow and the cost to upgrade the infrastructure to get
high through put I/O is expensive and no one has ever offered the
budget to resolve that problem. The fact of the matter with this is
that we now produce a lot of release artifacts and that takes time,
some of it can probably be run more in parallel, and possibly some
things run at different times but without a combination of investment
in infrastructure, and maybe even hacking and slashing of components
this is not simply a check box.

That's why the proposal is to skip an entire release to give us time to
work on it. If it was easy, that wouldn't be necessary.
There are of course at least *two* possible solutions to "there's lots
of I/O". One is "throw money at making the I/O faster", the other is
"figure out how to do less I/O".
One hour is a *target*, not a *requirement*. I don't think anyone's
going to consider the time wasted if we wind up getting it down to two
hours instead of one.
(Though personally I'd like ten minutes. And two unicorns.)

Just so.

--
Paul
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists

[Peter Robinson]

Post by Paul Frields

Post by Peter Robinson
On Mon, Nov 26, 2018 at 6:53 PM Adam Williamson

Post by Adam Williamson

Post by Peter Robinson

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)

Can I have a unicorn? Everyone wants this bug absolutely no one has
done analysis.

I just don't believe this is true. I'm pretty sure Dennis and Kevin
have both looked into it before.

Yes, I mean recently, I was involved in the last time we attempted
this, and there was a number of things identified but quite a bit has
changed since that last happened.

One of the large factors -- which I think was noted in the page -- was
RPM scriptlets. Stephen Gallagher met with WIll Woods and James Antill
just recently to talk about this. The time savings was substantial.
Combining that with reducing the size of "speculative composes" that

There's no facts or details, to quote "The reported speed increase for
transactions is substantial. This may solve (or greatly reduce) the
compose speed problem, TBD." but it doesn't report the context of the
"speed increase for transactions" .... like which transactions?
There's a lot of I/O heavy processes in the compose, like createrepo,
that don't run any rpm tracsactions what so ever.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedor

[Stephen Gallagher]

Post by Peter Robinson
There's no facts or details, to quote "The reported speed increase for
transactions is substantial. This may solve (or greatly reduce) the
compose speed problem, TBD." but it doesn't report the context of the
"speed increase for transactions" .... like which transactions?
There's a lot of I/O heavy processes in the compose, like createrepo,
that don't run any rpm tracsactions what so ever.

Paul was referring to image creation here. We should be able to get a
fairly significant (at least one order of magnitude) speed-up on
creating any image that runs an RPM installation transaction. I'm not
sure how much that will save in the *whole* compose process, but as
it's an identified spot where we know how to optimize it, we should do
so.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org

[Peter Robinson]

Post by Stephen Gallagher

Post by Peter Robinson
There's no facts or details, to quote "The reported speed increase for
transactions is substantial. This may solve (or greatly reduce) the
compose speed problem, TBD." but it doesn't report the context of the
"speed increase for transactions" .... like which transactions?
There's a lot of I/O heavy processes in the compose, like createrepo,
that don't run any rpm tracsactions what so ever.

Paul was referring to image creation here. We should be able to get a
fairly significant (at least one order of magnitude) speed-up on
creating any image that runs an RPM installation transaction. I'm not
sure how much that will save in the *whole* compose process, but as
it's an identified spot where we know how to optimize it, we should do
so.

But ultimately removing rpm transactions is an optimisation to rpm
which, when it lands, the compose plus numerous other compoents that
use that code path will just consume and at that point get the
benefits of, it's not a reason to skip a release nor is it going to be
a revolution that gets the composes down to an hour. I don't believe
that that alone is and order of magnitude change, do you have any
actual hard figured to back the statement up?
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org

[Stephen Gallagher]

Post by Peter Robinson

Post by Stephen Gallagher

Post by Peter Robinson
There's no facts or details, to quote "The reported speed increase for
transactions is substantial. This may solve (or greatly reduce) the
compose speed problem, TBD." but it doesn't report the context of the
"speed increase for transactions" .... like which transactions?
There's a lot of I/O heavy processes in the compose, like createrepo,
that don't run any rpm tracsactions what so ever.

Paul was referring to image creation here. We should be able to get a
fairly significant (at least one order of magnitude) speed-up on
creating any image that runs an RPM installation transaction. I'm not
sure how much that will save in the *whole* compose process, but as
it's an identified spot where we know how to optimize it, we should do
so.

But ultimately removing rpm transactions is an optimisation to rpm
which, when it lands, the compose plus numerous other compoents that
use that code path will just consume and at that point get the
benefits of, it's not a reason to skip a release nor is it going to be
a revolution that gets the composes down to an hour. I don't believe
that that alone is and order of magnitude change, do you have any
actual hard figured to back the statement up?

Paul overstated the general importance of the scriptlet work. As I
said above, it will affect the RPM transactions. Will Woods has
numbers on the actual performance gains that I'm sure he'll share on
this thread at some point. I said above "I'm not sure how much that
will save in the *whole* compose process" and I meant that. I'm not
advocating that we postpone F31 solely for these changes. At the same
time, let's be careful not to slip into a "perfect is the enemy of
good" situation where we start critiquing individual solutions as not
having enough of an effect.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedo

[Matthew Miller]

Post by Peter Robinson
transactions is substantial. This may solve (or greatly reduce) the
compose speed problem, TBD." but it doesn't report the context of the
"speed increase for transactions" .... like which transactions?
There's a lot of I/O heavy processes in the compose, like createrepo,
that don't run any rpm tracsactions what so ever.

It seems like there's a lot of room to optimize those things too. For
example, extract the needed metadata into a database and update that at
build time rather than compose time, and run createrepo against that instead
of rpms directly.



--
Matthew Miller
<***@fedoraproject.org>
Fedora Project Leader
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archive

[Peter Robinson]

Post by Matthew Miller

Post by Peter Robinson
transactions is substantial. This may solve (or greatly reduce) the
compose speed problem, TBD." but it doesn't report the context of the
"speed increase for transactions" .... like which transactions?
There's a lot of I/O heavy processes in the compose, like createrepo,
that don't run any rpm tracsactions what so ever.

It seems like there's a lot of room to optimize those things too. For
example, extract the needed metadata into a database and update that at
build time rather than compose time, and run createrepo against that instead
of rpms directly.

Completely agree there's a LOT of improvement that can be done, but I
don't see why that development work cannot be done in parallel and put
it into production when each of the various different components are
ready to go. Ultimately a lot of those sort of ways of optimising
things are things that have been known for some time but no one has
actually stepped up to do the dev work and it it into production
shape. Stopping the standard distro work won't miraculously make that
other work happen and appear.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedo

[Josh Boyer]

Post by Peter Robinson

Post by Matthew Miller

Post by Peter Robinson
transactions is substantial. This may solve (or greatly reduce) the
compose speed problem, TBD." but it doesn't report the context of the
"speed increase for transactions" .... like which transactions?
There's a lot of I/O heavy processes in the compose, like createrepo,
that don't run any rpm tracsactions what so ever.

It seems like there's a lot of room to optimize those things too. For
example, extract the needed metadata into a database and update that at
build time rather than compose time, and run createrepo against that instead
of rpms directly.

Completely agree there's a LOT of improvement that can be done, but I
don't see why that development work cannot be done in parallel and put
it into production when each of the various different components are

Because the people that would be tasked with doing the development are
also tasked with cranking out the release. It's a "need more people
problem".

Post by Peter Robinson
ready to go. Ultimately a lot of those sort of ways of optimising
things are things that have been known for some time but no one has
actually stepped up to do the dev work and it it into production
shape. Stopping the standard distro work won't miraculously make that
other work happen and appear.

Well, I kind of disagree. Barring magical people that appear out of
the woodwork that know how the tools work and how they need to be
improved, I don't see an alternative. Not doing a release to focus on
our tech debt seems like a good tradeoff. If there are others that
really WANT to continue cranking the release with the tools as they
are today... that might be something that could be pursued. It would
still require net new contributors to a critical area.

josh
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archiv

[Dennis Gilmore]

Post by Josh Boyer

On Mon, Nov 26, 2018 at 9:48 PM Matthew Miller <

Post by Matthew Miller

Post by Peter Robinson
transactions is substantial. This may solve (or greatly reduce) the
compose speed problem, TBD." but it doesn't report the context of the
"speed increase for transactions" .... like which transactions?
There's a lot of I/O heavy processes in the compose, like createrepo,
that don't run any rpm tracsactions what so ever.

It seems like there's a lot of room to optimize those things too. For
example, extract the needed metadata into a database and update that at
build time rather than compose time, and run createrepo against that instead
of rpms directly.

Completely agree there's a LOT of improvement that can be done, but I
don't see why that development work cannot be done in parallel and put
it into production when each of the various different components are

Because the people that would be tasked with doing the development are
also tasked with cranking out the release. It's a "need more people
problem".

This is no longer entirely true, development of the compose tools is
done entirely by PNT DevOps today. Bodhi and other tools in parts of
the delivery pipeline are developed by people in Fedora Engineering who
also have other tasks to do.

Post by Josh Boyer

ready to go. Ultimately a lot of those sort of ways of optimising
things are things that have been known for some time but no one has
actually stepped up to do the dev work and it it into production
shape. Stopping the standard distro work won't miraculously make that
other work happen and appear.

Well, I kind of disagree. Barring magical people that appear out of
the woodwork that know how the tools work and how they need to be
improved, I don't see an alternative. Not doing a release to focus on
our tech debt seems like a good tradeoff. If there are others that
really WANT to continue cranking the release with the tools as they
are today... that might be something that could be pursued. It would
still require net new contributors to a critical area.

We would need to engage with the team working on the compose tools and
see what time they can dedicate to meeting Fedora's needs. Likely a
end state needs to be defined. then the work can be scoped.

Dennis

[Paul Frields]

Post by Dennis Gilmore

Post by Josh Boyer
Because the people that would be tasked with doing the development are
also tasked with cranking out the release. It's a "need more people problem".

This is no longer entirely true, development of the compose tools is
done entirely by PNT DevOps today. Bodhi and other tools in parts of
the delivery pipeline are developed by people in Fedora Engineering who
also have other tasks to do.

Post by Josh Boyer

Post by Peter Robinson
ready to go. Ultimately a lot of those sort of ways of optimising
things are things that have been known for some time but no one has
actually stepped up to do the dev work and it it into production
shape. Stopping the standard distro work won't miraculously make that
other work happen and appear.

Well, I kind of disagree. Barring magical people that appear out of
the woodwork that know how the tools work and how they need to be
improved, I don't see an alternative. Not doing a release to focus on
our tech debt seems like a good tradeoff. If there are others that
really WANT to continue cranking the release with the tools as they
are today... that might be something that could be pursued. It would
still require net new contributors to a critical area.

We would need to engage with the team working on the compose tools and
see what time they can dedicate to meeting Fedora's needs. Likely a
end state needs to be defined. then the work can be scoped.

There are a lot of assumptions wrapped up in these statements. I'm not
sure we should be assuming that delivery mechanisms ought to all be
tied to internal Red Hat teams. This might be a good opportunity for
us (meaning people variously assigned in both places) to think about
whether we should try to decouple more of our delivery side (compose,
push, etc.), while keeping a tight bond at the build/test side.

The customers RH serves have specific expectations, and in part that
dictates how delivery tooling is done. Binding the community to that
may be counterproductive. This is especially true now that RHEL 8 Beta
is out -- it's the perfect time for us to be rethinking at that level.

--
Paul
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject

[Josh Boyer]

Post by Paul Frields

Post by Dennis Gilmore

Post by Josh Boyer
Because the people that would be tasked with doing the development are
also tasked with cranking out the release. It's a "need more people problem".

This is no longer entirely true, development of the compose tools is
done entirely by PNT DevOps today. Bodhi and other tools in parts of
the delivery pipeline are developed by people in Fedora Engineering who
also have other tasks to do.

Post by Josh Boyer

Post by Peter Robinson
ready to go. Ultimately a lot of those sort of ways of optimising
things are things that have been known for some time but no one has
actually stepped up to do the dev work and it it into production
shape. Stopping the standard distro work won't miraculously make that
other work happen and appear.

Well, I kind of disagree. Barring magical people that appear out of
the woodwork that know how the tools work and how they need to be
improved, I don't see an alternative. Not doing a release to focus on
our tech debt seems like a good tradeoff. If there are others that
really WANT to continue cranking the release with the tools as they
are today... that might be something that could be pursued. It would
still require net new contributors to a critical area.

We would need to engage with the team working on the compose tools and
see what time they can dedicate to meeting Fedora's needs. Likely a
end state needs to be defined. then the work can be scoped.

There are a lot of assumptions wrapped up in these statements. I'm not
sure we should be assuming that delivery mechanisms ought to all be
tied to internal Red Hat teams. This might be a good opportunity for
us (meaning people variously assigned in both places) to think about
whether we should try to decouple more of our delivery side (compose,
push, etc.), while keeping a tight bond at the build/test side.

Sure. We still either need more people or to drop existing things to
cover that with the existing people.

Post by Paul Frields
The customers RH serves have specific expectations, and in part that
dictates how delivery tooling is done. Binding the community to that
may be counterproductive. This is especially true now that RHEL 8 Beta
is out -- it's the perfect time for us to be rethinking at that level.

There are already a number of tools that are shared across Fedora and
RHEL and I can tell you for a fact that many on the RHEL side would
like to see the same or similar improvements that have been mentioned
here. Let's start scoping the problems and solutions before we decide
Fedora should go off and do it's own thing (again).

I'm interested in evolving more than just tooling. I continue to want
to see Fedora and CentOS coming together more closely, so that we can
leverage both of the amazing communities these projects provide. If
we do that, we should be scoping how to do release tooling effectively
across both Fedora and CentOS, while still keeping RHEL in mind as
well. Having 3 (or more) different implementations of compose and
release tooling seems ridiculous. I'd like to get it down to a
community version and then have RHEL additions for product specific
things if possible, driven by RHEL stakeholders. That way we get an
actual upstream/downstream relationship with our tooling in addition
to our distro.

josh
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraprojec

[Dennis Gilmore]

Post by Paul Frields

Post by Dennis Gilmore

Post by Josh Boyer
Because the people that would be tasked with doing the
development are
also tasked with cranking out the release. It's a "need more people problem".

This is no longer entirely true, development of the compose tools is
done entirely by PNT DevOps today. Bodhi and other tools in parts of
the delivery pipeline are developed by people in Fedora Engineering who
also have other tasks to do.

Post by Josh Boyer

Post by Peter Robinson
ready to go. Ultimately a lot of those sort of ways of
optimising
things are things that have been known for some time but no one has
actually stepped up to do the dev work and it it into
production
shape. Stopping the standard distro work won't miraculously
make
that
other work happen and appear.

Well, I kind of disagree. Barring magical people that appear out of
the woodwork that know how the tools work and how they need to be
improved, I don't see an alternative. Not doing a release to focus on
our tech debt seems like a good tradeoff. If there are others that
really WANT to continue cranking the release with the tools as they
are today... that might be something that could be pursued. It would
still require net new contributors to a critical area.

We would need to engage with the team working on the compose tools and
see what time they can dedicate to meeting Fedora's needs. Likely a
end state needs to be defined. then the work can be scoped.

There are a lot of assumptions wrapped up in these statements. I'm not
sure we should be assuming that delivery mechanisms ought to all be
tied to internal Red Hat teams. This might be a good opportunity for
us (meaning people variously assigned in both places) to think about
whether we should try to decouple more of our delivery side (compose,
push, etc.), while keeping a tight bond at the build/test side.

There is a lot of assumptions in your assumptions, language is hard :),
I was trying to outline where things are at not at all how things
should be, sorry I was not clear enough there. My main goal was
pointing out changes from where things were 5 years ago.

Post by Paul Frields
The customers RH serves have specific expectations, and in part that
dictates how delivery tooling is done. Binding the community to that
may be counterproductive. This is especially true now that RHEL 8 Beta
is out -- it's the perfect time for us to be rethinking at that level.

Compose tooling is largely shared, delivery tooling is not, though
there is room for improvement and convergence on both sides. We should
also consider CentOS and how we can share tooling, process and
resources, they still use something else entirely. Everything used in
Fedora is developed in the open and can be contributed to by anyone.
We have been talking about making the whole process flexible and
dynamic for years. I know internally they would like to have many of
the same things Fedora wants.

Dennis

[Paul Frields]

On Tue, Nov 27, 2018 at 9:50 AM Dennis Gilmore <***@ausil.us> wrote:
[...snip...]

Post by Dennis Gilmore

Post by Paul Frields
The customers RH serves have specific expectations, and in part that
dictates how delivery tooling is done. Binding the community to that
may be counterproductive. This is especially true now that RHEL 8 Beta
is out -- it's the perfect time for us to be rethinking at that level.

Compose tooling is largely shared, delivery tooling is not, though
there is room for improvement and convergence on both sides. We should
also consider CentOS and how we can share tooling, process and
resources, they still use something else entirely. Everything used in
Fedora is developed in the open and can be contributed to by anyone.
We have been talking about making the whole process flexible and
dynamic for years. I know internally they would like to have many of
the same things Fedora wants.

Yeah, it sounds like we're aiming in the same direction here. This is
part of what I mean by "decomposing the compose." We have one rather
monolithic process right now, all or nothing. And it does so much that
it's impossible to get any fast results from it. That doesn't mean it
doesn't work well for what it does. But it's not very flexible, as you
pointed out.

With a more flexible compose we could choose to do smaller tasks (as
well as capitalize on other time savings) for something like a
speculative testing build. That kind of build could be used to perform
integration testing as well as other tasks (automated, not manual!).
We could even potentially test against much of the repetitive,
time-consuming matrices manually (and heroically) run by QA. Then only
choose to do more extensive tasks based on that being "green."

--
Paul
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedor

[Peter Robinson]

Post by Paul Frields
[...snip...]

Post by Dennis Gilmore

Post by Paul Frields
The customers RH serves have specific expectations, and in part that
dictates how delivery tooling is done. Binding the community to that
may be counterproductive. This is especially true now that RHEL 8 Beta
is out -- it's the perfect time for us to be rethinking at that level.

Compose tooling is largely shared, delivery tooling is not, though
there is room for improvement and convergence on both sides. We should
also consider CentOS and how we can share tooling, process and
resources, they still use something else entirely. Everything used in
Fedora is developed in the open and can be contributed to by anyone.
We have been talking about making the whole process flexible and
dynamic for years. I know internally they would like to have many of
the same things Fedora wants.

Yeah, it sounds like we're aiming in the same direction here. This is
part of what I mean by "decomposing the compose." We have one rather
monolithic process right now, all or nothing. And it does so much that
it's impossible to get any fast results from it. That doesn't mean it
doesn't work well for what it does. But it's not very flexible, as you
pointed out.

Actually amusingly if you look back through the history post F-21 and
the massive delays that happened due to various non monolithic
processes one of the outcomes of the "get a release done with
Fedora.next" was to be able to do full composes every day because
prior to that we only did newRepo, a network install and live/cloud
images and often we'd get to Alpha TC1 right after branching to
discover a whole bunch of stuff was completely broken. We finally got
this with the move to "pungi 4" in Fedora 24, and in the intervening
releases we had huge amounts of complaints because rel-eng couldn't
provide all release artifacts to be pushed through CI/CD so
regressions could be fixed instantaneously. Yes, I'm aware things
change, and I often push them, but it's worth knowing the history how
we got to where we are!

Post by Paul Frields
With a more flexible compose we could choose to do smaller tasks (as
well as capitalize on other time savings) for something like a
speculative testing build. That kind of build could be used to perform
integration testing as well as other tasks (automated, not manual!).

Yes, and in some regards there's already parts of the project doing
this, the TWA release runs like this, as does the IoT release and I
mentioned above ways to break some of the bits up more.

Post by Paul Frields
We could even potentially test against much of the repetitive,
time-consuming matrices manually (and heroically) run by QA. Then only
choose to do more extensive tasks based on that being "green."

Completely agree, but there's quite a bit of work to be done to get to
that unicorn but none of the process to get there has been documented
or even proposed in this thread, it's not easy (believe me a lot of
people have looked over the years I've been involved in rel-eng) and
most of what I've seen of actual concrete means to get there won't get
us anywhere close to that, while yes it will still improve it.

Peter
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedoraproject.or

[Adam Williamson]

Post by Peter Robinson

Post by Paul Frields
[...snip...]

Post by Dennis Gilmore

Post by Paul Frields
The customers RH serves have specific expectations, and in part that
dictates how delivery tooling is done. Binding the community to that
may be counterproductive. This is especially true now that RHEL 8 Beta
is out -- it's the perfect time for us to be rethinking at that level.

Compose tooling is largely shared, delivery tooling is not, though
there is room for improvement and convergence on both sides. We should
also consider CentOS and how we can share tooling, process and
resources, they still use something else entirely. Everything used in
Fedora is developed in the open and can be contributed to by anyone.
We have been talking about making the whole process flexible and
dynamic for years. I know internally they would like to have many of
the same things Fedora wants.

Yeah, it sounds like we're aiming in the same direction here. This is
part of what I mean by "decomposing the compose." We have one rather
monolithic process right now, all or nothing. And it does so much that
it's impossible to get any fast results from it. That doesn't mean it
doesn't work well for what it does. But it's not very flexible, as you
pointed out.

Actually amusingly if you look back through the history post F-21 and
the massive delays that happened due to various non monolithic
processes one of the outcomes of the "get a release done with
Fedora.next" was to be able to do full composes every day because
prior to that we only did newRepo, a network install and live/cloud
images and often we'd get to Alpha TC1 right after branching to
discover a whole bunch of stuff was completely broken. We finally got
this with the move to "pungi 4" in Fedora 24, and in the intervening
releases we had huge amounts of complaints because rel-eng couldn't
provide all release artifacts to be pushed through CI/CD so
regressions could be fixed instantaneously. Yes, I'm aware things
change, and I often push them, but it's worth knowing the history how
we got to where we are!

Sure, I know about this (I did quite a bit of work around it). But I
don't see any kind of inconsistency or anything here. Yes, we needed to
ability to compose the whole distro in a saner and more organized way,
which is what Pungi 4 did, which was a significant improvement. But it
would *also* be extremely useful to be able to do smaller, faster
subsets of the compose.

It's not like that's what we were doing before, after all. As you say,
we still only did it *nightly*. And since it still involved a newRepo,
it still took forever.

Post by Peter Robinson

Post by Paul Frields
With a more flexible compose we could choose to do smaller tasks (as
well as capitalize on other time savings) for something like a
speculative testing build. That kind of build could be used to perform
integration testing as well as other tasks (automated, not manual!).

Yes, and in some regards there's already parts of the project doing
this, the TWA release runs like this, as does the IoT release and I
mentioned above ways to break some of the bits up more.

I know about this too, and mentioned it in my initial mail in this
subthread. But it's all been done in a manual, ad-hoc way: we just
write entirely new Pungi config files for each new compose profile we
come up with. There's no notion of modularity, of being able to
actually define the individual elements and easily combine and
recombine them. There's no notion of inputs mapped to outputs, either,
really. I don't think this approach really scales much further than
we're currently doing it; I don't think it'd be practical to (for
instance) manually split the big 'Fedora' compose into 'Fedora-Live',
'Fedora-ISO', 'Fedora-ARM-Disk', etc. etc. etc. Trying to do that *just
along the same lines as we have created the existing 'new' composes*
would probably give us an unmaintainable mess.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedora

[Fabio Valentini]

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post.

Hi Matthew,

I initially basically ignored the previous E-Mail, because it didn't
come from a person I knew, and - from quickly glancing over the
contents - read like marketing fluff. I should have dug deeper there,
but we all only have limited time.

Post by Matthew Miller
Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level

This is definitely getting more attention. Maybe the initial E-Mail
should have had a less bland Subject line, too ;)

Post by Matthew Miller
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)
* really actually for real gated Rawhide
* better CI pipeline tests for everything
* better tooling for non-base deliverables
* better metrics for everything

Those all sound like good things to work on.

Post by Matthew Miller
* define a base platform -- Red Hat wants to focus resources here
#3 (...) Define a small, draft set of content for the Platform which can iterate over time

What exactly does that mean? Moving to something two-tiered like the
ubuntu model (main / universe) - which I think is a terrible solution?
(And what would be the names for this in fedora - "core" and "extras"?
(pun definitely intended))

Also, I find it funny that - somehow - tons of resources would be
available for "better CI pipeline tests for everything" (!), but we'd
have to "focus resources" when it comes to the fedora "platform"
itself.

I hope my response doesn't read overly negative or critical - but I
think some fedora contributors (including me) have been weary of
something like this ("focusing resources") since the IBM news got
announced.

Fabio

Post by Matthew Miller
Please read https://fedoraproject.org/wiki/Objectives/Lifecycle/Problem_statements
and comment in this thread.
--
Matthew Miller
Fedora Project Leader
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproje

[Adam Williamson]

Post by Fabio Valentini
Also, I find it funny that - somehow - tons of resources would be
available for "better CI pipeline tests for everything" (!), but we'd
have to "focus resources" when it comes to the fedora "platform"
itself.

These two sort of go together, I think: we don't truly expect to have
'better CI pipeline tests for EVERYTHING', as in *every package in the
distro*, but rather one advantage of somehow defining the 'Platform' is
that it gives us a *smaller* set of bits to focus the CI work on.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archive

[drago01]

Post by Matthew Miller

Post by Paul Frields
Here's the summary from the page, which proposes we pause the release

I know it was a big time-off holiday week in the US, but I expected a little
more interest in this post. Perhaps it seemed like too much text to digest
along with turkey and stuffing. :) I'm highlighting it with a subject
reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)
* really actually for real gated Rawhide
* better CI pipeline tests for everything
* define a base platform -- Red Hat wants to focus resources here
* better tooling for non-base deliverables
* better metrics for everything
Please read https://fedoraproject.org/wiki/Objectives/Lifecycle/
Problem_statements
and comment in this thread.

I am not really convinced that delaying the release for that makes sense.

It's not like people will stop updating packages or that existing release
tools will magically disappear (they have to work to keep existing versions
running).

So people can work on those improvement and bring them in when ready but
stopping everything for that is overkill.

[Miroslav Suchý]

Post by Matthew Miller
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)
* really actually for real gated Rawhide
* better CI pipeline tests for everything
* define a base platform -- Red Hat wants to focus resources here
* better tooling for non-base deliverables
* better metrics for everything

My experience is that things are fixed when we are in real time stress. When we have more time, then people (including
me) are trying to develop/invent new things.

I have a counter proposal: Make the release every week or month - this will force us to automate all release processes :)

Miroslav
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@

[Paul Frields]

Post by Miroslav Suchý

Post by Matthew Miller
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)
* really actually for real gated Rawhide
* better CI pipeline tests for everything
* define a base platform -- Red Hat wants to focus resources here
* better tooling for non-base deliverables
* better metrics for everything

My experience is that things are fixed when we are in real time stress. When we have more time, then people (including
me) are trying to develop/invent new things.
I have a counter proposal: Make the release every week or month - this will force us to automate all release processes :)

It's funny you mention that, because I could foresee one outcome of
focusing on a core platform being that we do a more frequent release.
If we can effectively machine-test more of that platform, we could do
releases on a timescale of weeks instead of only twice a year. The
Atomic WG does this already. This probably makes more sense for an
ostree based release than constantly spinning large, multi-GB ISOs.

--
Paul
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/li

[Kevin Kofler]

Post by Matthew Miller
I know it was a big time-off holiday week in the US, but I expected a
little more interest in this post. Perhaps it seemed like too much text to
digest along with turkey and stuffing. :) I'm highlighting it with a
subject reflecting the big, direct impact, and here's some other top-level
* embrace Taiga (an open source kanban tool) for project planning
* fix the compose speed (target: one hour!)
* really actually for real gated Rawhide
* better CI pipeline tests for everything
* define a base platform -- Red Hat wants to focus resources here
* better tooling for non-base deliverables
* better metrics for everything

And why do we need to stop the presses to do that? This is entirely
orthogonal to distribution development and can be done in parallel.

Kevin Kofler
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedoraprojec

[Randy Barlow]

Post by Kevin Kofler
And why do we need to stop the presses to do that? This is entirely
orthogonal to distribution development and can be done in parallel.

The problem is that the people who do the work to get the release done
are the same people who would need to do the retooling work, so there
is a resource conflict.

[Brendan Conoboy]

On 11/16/18 7:50 AM, Paul Frields wrote:
[snip]

Post by Paul Frields
We should skip the F31 release cycle and leave F30 in place longer in
order to focus on improving the tooling and testing changes. These
tooling changes will improve the overall reliability of Fedora, and
will decrease the manual effort and complexities involved in producing
the distribution artifacts. Although we’ve done this before to make
“editions” happen, the intent is to track this multi-team effort more
actively so we can (1) use the time as well as possible, and (2) give
the work maximum transparency.

If there is going to be a pause F30 seems like a good place to do it:
New glibc, new compiler- and a full year for them to mature. It's a
nice basis for a stable platform. What would the update policy be for
this year- same as today? It seems like you're proposing this as a
one-time event to pay down technical debt, which is great, but would
you perhaps consider doing the same thing for F31, F32, etc? The
basic reasons for technical debt will continue- why not plan to
service the debt regularly?

--
Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.or

[Igor Gnatenko]

Because if we keep "no breaking updates in stable" policy, then Fedora
won't be "first anymore". You can do this only if rawhide will be more
popular between people.

Post by Brendan Conoboy
[snip]

Post by Paul Frields
We should skip the F31 release cycle and leave F30 in place longer in
order to focus on improving the tooling and testing changes. These
tooling changes will improve the overall reliability of Fedora, and
will decrease the manual effort and complexities involved in producing
the distribution artifacts. Although we’ve done this before to make
“editions” happen, the intent is to track this multi-team effort more
actively so we can (1) use the time as well as possible, and (2) give
the work maximum transparency.

New glibc, new compiler- and a full year for them to mature. It's a
nice basis for a stable platform. What would the update policy be for
this year- same as today? It seems like you're proposing this as a
one-time event to pay down technical debt, which is great, but would
you perhaps consider doing the same thing for F31, F32, etc? The
basic reasons for technical debt will continue- why not plan to
service the debt regularly?
--
Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc.
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

[Ralf Corsepius]

Post by Igor Gnatenko
Because if we keep "no breaking updates in stable" policy, then Fedora
won't be "first anymore".

Users perceive your "first" as unstable and unreliable. There are plenty
of examples of how e.g. FC30 was broken and still is.

One such example is you personal favorite: dnf.

Post by Igor Gnatenko
You can do this only if rawhide will be more
popular between people.

Rawhide will never be end-user usable. I personally consider rawhide
end-user usability to be a hoax.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorapr

[Igor Gnatenko]

Post by Ralf Corsepius

Post by Igor Gnatenko
Because if we keep "no breaking updates in stable" policy, then Fedora
won't be "first anymore".

Users perceive your "first" as unstable and unreliable. There are plenty
of examples of how e.g. FC30 was broken and still is.

Then it means that we have to do releases, otherwise we will stay with
one version with outdated software not just for 6 months but for one
year.

Post by Ralf Corsepius
One such example is you personal favorite: dnf.

Post by Igor Gnatenko
You can do this only if rawhide will be more
popular between people.

Rawhide will never be end-user usable. I personally consider rawhide
end-user usability to be a hoax.
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives

[Neal Gompa]

Post by Brendan Conoboy
[snip]

Post by Paul Frields
We should skip the F31 release cycle and leave F30 in place longer in
order to focus on improving the tooling and testing changes. These
tooling changes will improve the overall reliability of Fedora, and
will decrease the manual effort and complexities involved in producing
the distribution artifacts. Although we’ve done this before to make
“editions” happen, the intent is to track this multi-team effort more
actively so we can (1) use the time as well as possible, and (2) give
the work maximum transparency.

New glibc, new compiler- and a full year for them to mature. It's a
nice basis for a stable platform. What would the update policy be for
this year- same as today? It seems like you're proposing this as a
one-time event to pay down technical debt, which is great, but would
you perhaps consider doing the same thing for F31, F32, etc? The
basic reasons for technical debt will continue- why not plan to
service the debt regularly?

I'm going to have a longer post to reply about the overall topic
brought up over Turkey Day, but for this specific point, I think that
it would be a tremendous mistake if we allowed RHEL to influence us
into slowing down the core of Fedora (see, I said the "bad" word!).

In the past few years, I've been around the block a bit, being
involved in different distribution release philosophies and processes.
And one thing that is common among all of them is that "stuff rolls
downstream". That is, an action from the top part usually has
cascading effects downward.

Most of us don't consider the "core"/"base platform" of Fedora to be
the "top", but it is. All the decisions about how every package is
built and shipped is affected by this. It would be incredibly
dangerous for Fedora to take a RHEL-like approach to the core, because
it means that we're deciding that the core is not a place for
innovation and advancement.

Now, I'll admit I have a personal stake here: much of what *I* do
these days is in that part of the stack, and having that frozen would
kill my ability to participate in the development of the distribution.
But I'd like us to become less conservative instead of more so. I've
been playing with the RHEL 8 beta for a bit now, and I have a pretty
good idea what constitutes a "base" system in RHEL, and I'm afraid
that it would be terrible if we froze that for a year.

However, to the larger point about lifecycles and supporting hardware
makers with a Fedora LTS that works for 4 years, this could be a good
focus point for a new "Fedora Long-Term" WG that would work to take
selected releases of Fedora and stabilize them for a period of time.
This would be similar to the original Fedora Legacy and openSUSE
Evergreen[1] projects. An interesting twist here is that we could take
some inspiration from Debian on how they run their LTS and allow
direct sponsorship for releases to be supported past their original
life[2]. That means that under the auspices of the Fedora Project,
interested volunteers and commercial entities could come together and
maintain a release with security fixes past the 13 month life, for an
additional 2 years.

This approach has some advantages: notably only releases people are
interested it would qualify, and the work burden scales with interest
and sponsorship. The main disadvantage is that it won't happen "for
free", which I think is actually a good thing here.

Even Red Hat could participate in this program, though I fully expect
that they won't, since they have their hands full with Red Hat
Enterprise Linux. But it'd be nice if they did, because it could also
be an avenue to improve the transparency between Fedora and RHEL, and
give a more direct relationship (and potential for influencing
improvements to RHEL that we don't have today!). It would also make it
easier for the 22 other product projects that Red Hat has to use
Fedora as their upstream rather than CentOS, since some folks in those
communities complain Fedora is too fast for them (*hack*RDO*cough*).

[1]: https://en.opensuse.org/openSUSE:Evergreen
[2]: https://wiki.debian.org/LTS/

--
真実はいつも一つ！/ Always, there's only one truth!
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/deve

[Justin Forbes]

Post by Brendan Conoboy
[snip]

Post by Paul Frields
We should skip the F31 release cycle and leave F30 in place longer in
order to focus on improving the tooling and testing changes. These
tooling changes will improve the overall reliability of Fedora, and
will decrease the manual effort and complexities involved in producing
the distribution artifacts. Although we’ve done this before to make
“editions” happen, the intent is to track this multi-team effort more
actively so we can (1) use the time as well as possible, and (2) give
the work maximum transparency.

New glibc, new compiler- and a full year for them to mature. It's a
nice basis for a stable platform. What would the update policy be for
this year- same as today? It seems like you're proposing this as a
one-time event to pay down technical debt, which is great, but would
you perhaps consider doing the same thing for F31, F32, etc? The
basic reasons for technical debt will continue- why not plan to
service the debt regularly?

Long cycles have been done before, and will be done again, it has been
4 or 5 years since the last one. I think skipping to a yearly cadence
for every release isn't such a great idea. There are benefits to the
cadence we have, but I do think perhaps a plan to regularly do this
might be a good thing. Just as a ballpark, perhaps once every 5
releases. This would allow people to actually plan the big workloads
for this timeframe. F30 is a 1 year, the next is F35. Once F31 is
done, I can start planning the big project for F36. Of course it
could also be argued that we only do a long release when needed, but
this doesn't really allow us to plan as well, and doesn't set
expectations for the community in the same way.

Justin
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/l

[Ben Cotton]

Post by Justin Forbes
Long cycles have been done before, and will be done again, it has been
4 or 5 years since the last one. I think skipping to a yearly cadence
for every release isn't such a great idea. There are benefits to the
cadence we have, but I do think perhaps a plan to regularly do this
might be a good thing.

Agreed. Doing a long cycle is a tradeoff. In general, the six month
cycle works for what we're trying to do. But having the occasional
long cycle gives us some space to work on bigger projects or things
that can't be done while we're also trying to get a release out the
door. With unlimited funding and people, we wouldn't have to make this
tradeoff, but we do have limits.

I'm sympathetic to the argument that we should do it as needed, but my
preference is to schedule it. This allows our community to plan ahead,
both for the work to be done during the long cycle and also for things
that should get done before and after. Predictability is helpful.

If we do say, for example, every 8th release will be a long cycle
there's nothing that would prevent us from doing an unscheduled long
cycle if we have to. But it would have to be a very compelling reason
knowing that we'd have another long release coming up.

--
Ben Cotton
Fedora Program Manager
TZ=America/Indiana/Indianapolis
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives

[Brendan Conoboy]

Post by Ben Cotton

Post by Justin Forbes
Long cycles have been done before, and will be done again, it has been
4 or 5 years since the last one. I think skipping to a yearly cadence
for every release isn't such a great idea. There are benefits to the
cadence we have, but I do think perhaps a plan to regularly do this
might be a good thing.

Agreed. Doing a long cycle is a tradeoff. In general, the six month
cycle works for what we're trying to do. But having the occasional
long cycle gives us some space to work on bigger projects or things
that can't be done while we're also trying to get a release out the
door. With unlimited funding and people, we wouldn't have to make this
tradeoff, but we do have limits.
I'm sympathetic to the argument that we should do it as needed, but my
preference is to schedule it. This allows our community to plan ahead,
both for the work to be done during the long cycle and also for things
that should get done before and after. Predictability is helpful.
If we do say, for example, every 8th release will be a long cycle
there's nothing that would prevent us from doing an unscheduled long
cycle if we have to. But it would have to be a very compelling reason
knowing that we'd have another long release coming up.

The other way to slice it is having some parts of the distribution be
on a longer cycle and others on a shorter cycle. If there is a 1 year
gap between F30 and F31, perhaps the rules for what can be updated in
F30 could be revised to maintain the value people get from the 6 month
release cycle.

--
Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@li

[Owen Taylor]

Post by Brendan Conoboy
[snip]

Post by Paul Frields
We should skip the F31 release cycle and leave F30 in place longer in
order to focus on improving the tooling and testing changes. These
tooling changes will improve the overall reliability of Fedora, and
will decrease the manual effort and complexities involved in producing
the distribution artifacts. Although we’ve done this before to make
“editions” happen, the intent is to track this multi-team effort more
actively so we can (1) use the time as well as possible, and (2) give
the work maximum transparency.

New glibc, new compiler- and a full year for them to mature. It's a
nice basis for a stable platform. What would the update policy be for
this year- same as today? It seems like you're proposing this as a
one-time event to pay down technical debt, which is great, but would
you perhaps consider doing the same thing for F31, F32, etc? The
basic reasons for technical debt will continue- why not plan to
service the debt regularly?

If we go ahead and skip the F31 release, I see this as a (perhaps
necessary) desperation move - there is a small group of a dozen or so
people who would be key to improving our release processes, but those
people are always busy making releases.

But for the next thousand or so Fedora developers, the release cycle
is actually not a big deal - not something that takes much of their
time - and it gives them a regular place to land feature work. And
Fedora users appreciate a timely updated operating system (without
having random rebases trickle in.)

In other words, the "technical debt" we are trying to solve here is
not project wide and doesn't justify slowing down the whole project
permanently.

- Owen
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedorapr

[Josh Boyer]

Post by Owen Taylor

Post by Brendan Conoboy
[snip]

Post by Paul Frields
We should skip the F31 release cycle and leave F30 in place longer in
order to focus on improving the tooling and testing changes. These
tooling changes will improve the overall reliability of Fedora, and
will decrease the manual effort and complexities involved in producing
the distribution artifacts. Although we’ve done this before to make
“editions” happen, the intent is to track this multi-team effort more
actively so we can (1) use the time as well as possible, and (2) give
the work maximum transparency.

New glibc, new compiler- and a full year for them to mature. It's a
nice basis for a stable platform. What would the update policy be for
this year- same as today? It seems like you're proposing this as a
one-time event to pay down technical debt, which is great, but would
you perhaps consider doing the same thing for F31, F32, etc? The
basic reasons for technical debt will continue- why not plan to
service the debt regularly?

If we go ahead and skip the F31 release, I see this as a (perhaps
necessary) desperation move - there is a small group of a dozen or so
people who would be key to improving our release processes, but those
people are always busy making releases.
But for the next thousand or so Fedora developers, the release cycle
is actually not a big deal - not something that takes much of their
time - and it gives them a regular place to land feature work. And
Fedora users appreciate a timely updated operating system (without
having random rebases trickle in.)

It's not a big deal because they don't participate in making the
release nor do they really know about all the duct tape and bailing
wire holding all the machinery in place. Just like most of them don't
appreciate the heroics involved from Fedora QA and rel-eng and the
dozen or so people that regularly go well beyond the extra mile to get
it out the door. This isn't done out of malice on their part. It's
mostly that they just don't see it.

Post by Owen Taylor
In other words, the "technical debt" we are trying to solve here is
not project wide and doesn't justify slowing down the whole project
permanently.

I completely disagree. Our release process and tooling is built on
heroism and tech debt. At some point, that is going to cause
significant burnout and then people will just wonder what happened
when those people stop doing releases. I think it's better to raise
awareness at the project level, and build something that is
sustainable rather than predicated on "small group of people killing
themselves for the greater good". That starts with individual package
CI gating, and goes all the way through integration testing between
packages in a gated manner, into how we actually *create* all of that
plus the things we deem worth of releasing.

josh
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedor

[Owen Taylor]

Post by Josh Boyer

Post by Owen Taylor
But for the next thousand or so Fedora developers, the release cycle
is actually not a big deal - not something that takes much of their
time - and it gives them a regular place to land feature work. And
Fedora users appreciate a timely updated operating system (without
having random rebases trickle in.)

It's not a big deal because they don't participate in making the
release nor do they really know about all the duct tape and bailing
wire holding all the machinery in place. Just like most of them don't
appreciate the heroics involved from Fedora QA and rel-eng and the
dozen or so people that regularly go well beyond the extra mile to get
it out the door. This isn't done out of malice on their part. It's
mostly that they just don't see it.

If we distribute this work out across the 1000 developers, we'll make
life better for heroes, and it's *still* not going to be a big deal
for the 1000 developers.

Post by Josh Boyer

Post by Owen Taylor
In other words, the "technical debt" we are trying to solve here is
not project wide and doesn't justify slowing down the whole project
permanently.

I completely disagree. Our release process and tooling is built on
heroism and tech debt. At some point, that is going to cause
significant burnout and then people will just wonder what happened
when those people stop doing releases. I think it's better to raise
awareness at the project level, and build something that is
sustainable rather than predicated on "small group of people killing
themselves for the greater good". That starts with individual package
CI gating, and goes all the way through integration testing between
packages in a gated manner, into how we actually *create* all of that
plus the things we deem worth of releasing.

I think you are misunderstanding me somehow. I'm 100% on-board with
improving the processes, catching compose problems in an automated and
early fashion, making developers fix their own problems, and generally
making releases a less heroic process. And if that requires a
temporary cadence chance to get the retooling done, then it's worth
it.

But Brendan's proposal to permanently slow down the cadence seems to
make the assumption that the current release cycle is a heroic effort
for the entire project - that we're moving too fast to stay on our
feet. And I don't think that's the case.

Owen
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists

[Clement Verna]

Post by Owen Taylor

Post by Josh Boyer

Post by Owen Taylor
But for the next thousand or so Fedora developers, the release cycle
is actually not a big deal - not something that takes much of their
time - and it gives them a regular place to land feature work. And
Fedora users appreciate a timely updated operating system (without
having random rebases trickle in.)

It's not a big deal because they don't participate in making the
release nor do they really know about all the duct tape and bailing
wire holding all the machinery in place. Just like most of them don't
appreciate the heroics involved from Fedora QA and rel-eng and the
dozen or so people that regularly go well beyond the extra mile to get
it out the door. This isn't done out of malice on their part. It's
mostly that they just don't see it.

If we distribute this work out across the 1000 developers, we'll make
life better for heroes, and it's *still* not going to be a big deal
for the 1000 developers.

I think this is another important problem we have, it is very
difficult for someone interested to help in these domains. So we end
up with a dozen of persons being heroic because they are actually the
one that have the correct permissions and knowledge to do most of the
work. We should definitely look at solving this problem.

Post by Owen Taylor

Post by Josh Boyer

Post by Owen Taylor
In other words, the "technical debt" we are trying to solve here is
not project wide and doesn't justify slowing down the whole project
permanently.

I completely disagree. Our release process and tooling is built on
heroism and tech debt. At some point, that is going to cause
significant burnout and then people will just wonder what happened
when those people stop doing releases. I think it's better to raise
awareness at the project level, and build something that is
sustainable rather than predicated on "small group of people killing
themselves for the greater good". That starts with individual package
CI gating, and goes all the way through integration testing between
packages in a gated manner, into how we actually *create* all of that
plus the things we deem worth of releasing.

I think you are misunderstanding me somehow. I'm 100% on-board with
improving the processes, catching compose problems in an automated and
early fashion, making developers fix their own problems, and generally
making releases a less heroic process. And if that requires a
temporary cadence chance to get the retooling done, then it's worth
it.
But Brendan's proposal to permanently slow down the cadence seems to
make the assumption that the current release cycle is a heroic effort
for the entire project - that we're moving too fast to stay on our
feet. And I don't think that's the case.
Owen
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel

[Josh Boyer]

Post by Owen Taylor

Post by Josh Boyer

Post by Owen Taylor
But for the next thousand or so Fedora developers, the release cycle
is actually not a big deal - not something that takes much of their
time - and it gives them a regular place to land feature work. And
Fedora users appreciate a timely updated operating system (without
having random rebases trickle in.)

It's not a big deal because they don't participate in making the
release nor do they really know about all the duct tape and bailing
wire holding all the machinery in place. Just like most of them don't
appreciate the heroics involved from Fedora QA and rel-eng and the
dozen or so people that regularly go well beyond the extra mile to get
it out the door. This isn't done out of malice on their part. It's
mostly that they just don't see it.

If we distribute this work out across the 1000 developers, we'll make
life better for heroes, and it's *still* not going to be a big deal
for the 1000 developers.

I would say it will be a learning curve and there will be complaints
because it's change and people seem to only want change if it's the
change they want or otherwise doesn't impact them. That's going to be
a big deal to some :) Generally though, yes. We need to distribute
the burden to the package sets and whomever maintains them.

Post by Owen Taylor

Post by Josh Boyer

Post by Owen Taylor
In other words, the "technical debt" we are trying to solve here is
not project wide and doesn't justify slowing down the whole project
permanently.

I completely disagree. Our release process and tooling is built on
heroism and tech debt. At some point, that is going to cause
significant burnout and then people will just wonder what happened
when those people stop doing releases. I think it's better to raise
awareness at the project level, and build something that is
sustainable rather than predicated on "small group of people killing
themselves for the greater good". That starts with individual package
CI gating, and goes all the way through integration testing between
packages in a gated manner, into how we actually *create* all of that
plus the things we deem worth of releasing.

I think you are misunderstanding me somehow. I'm 100% on-board with
improving the processes, catching compose problems in an automated and
early fashion, making developers fix their own problems, and generally
making releases a less heroic process. And if that requires a
temporary cadence chance to get the retooling done, then it's worth
it.

Ah, good!

Post by Owen Taylor
But Brendan's proposal to permanently slow down the cadence seems to
make the assumption that the current release cycle is a heroic effort
for the entire project - that we're moving too fast to stay on our
feet. And I don't think that's the case.

That would be a concern to me as well, assuming we stuck with a
*single* cadence and a *single* platform. With the lifecycle
Objective, aren't we targeting multiple cadences and lifecycles? I
mean, we have this today with Fedora Atomic/CoreOS vs. "regular"
Fedora. Maybe Brendan was thinking singular, but I would very much
like the ability in our tooling a process to have both the faster
moving cadence of today AND a slower moving one. That's another
reason I view a temporary halt as worthwhile. If done correctly, it
enables Fedora to fill more usecases and lifecycles than we could ever
accomplish with a single release.

josh
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/a

[Brendan Conoboy]

Post by Josh Boyer

Post by Owen Taylor

Post by Josh Boyer

Post by Owen Taylor
But for the next thousand or so Fedora developers, the release cycle
is actually not a big deal - not something that takes much of their
time - and it gives them a regular place to land feature work. And
Fedora users appreciate a timely updated operating system (without
having random rebases trickle in.)

It's not a big deal because they don't participate in making the
release nor do they really know about all the duct tape and bailing
wire holding all the machinery in place. Just like most of them don't
appreciate the heroics involved from Fedora QA and rel-eng and the
dozen or so people that regularly go well beyond the extra mile to get
it out the door. This isn't done out of malice on their part. It's
mostly that they just don't see it.

If we distribute this work out across the 1000 developers, we'll make
life better for heroes, and it's *still* not going to be a big deal
for the 1000 developers.

I would say it will be a learning curve and there will be complaints
because it's change and people seem to only want change if it's the
change they want or otherwise doesn't impact them. That's going to be
a big deal to some :) Generally though, yes. We need to distribute
the burden to the package sets and whomever maintains them.

Post by Owen Taylor

Post by Josh Boyer

Post by Owen Taylor
In other words, the "technical debt" we are trying to solve here is
not project wide and doesn't justify slowing down the whole project
permanently.

I completely disagree. Our release process and tooling is built on
heroism and tech debt. At some point, that is going to cause
significant burnout and then people will just wonder what happened
when those people stop doing releases. I think it's better to raise
awareness at the project level, and build something that is
sustainable rather than predicated on "small group of people killing
themselves for the greater good". That starts with individual package
CI gating, and goes all the way through integration testing between
packages in a gated manner, into how we actually *create* all of that
plus the things we deem worth of releasing.

I think you are misunderstanding me somehow. I'm 100% on-board with
improving the processes, catching compose problems in an automated and
early fashion, making developers fix their own problems, and generally
making releases a less heroic process. And if that requires a
temporary cadence chance to get the retooling done, then it's worth
it.

Ah, good!

Post by Owen Taylor
But Brendan's proposal to permanently slow down the cadence seems to
make the assumption that the current release cycle is a heroic effort
for the entire project - that we're moving too fast to stay on our
feet. And I don't think that's the case.

That would be a concern to me as well, assuming we stuck with a
*single* cadence and a *single* platform. With the lifecycle
Objective, aren't we targeting multiple cadences and lifecycles? I
mean, we have this today with Fedora Atomic/CoreOS vs. "regular"
Fedora. Maybe Brendan was thinking singular, but I would very much
like the ability in our tooling a process to have both the faster
moving cadence of today AND a slower moving one. That's another
reason I view a temporary halt as worthwhile. If done correctly, it
enables Fedora to fill more usecases and lifecycles than we could ever
accomplish with a single release.

Josh has it: I am indeed thinking about how this works with the
lifecycle objective. When we think about things as all or nothing we
rapidly get into a zero sum game: Let's make this part of Fedora worse
to make this other part better. We don't have to do that, we can just
make Fedora better. That's what having multiple cadences and
lifecycles is all about. So it is important to talk about how the
rules would change if F30 takes a year, because if the rules don't
change it might diminish one of the great things about Fedora.

--
Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedorap

[Owen Taylor]

We can definitely talk about whether moving to a slower cadence for
certain parts of the base platform. But people don't judge Fedora on
how beautifully we maintain glibc and gcc - they mostly judge it by
installing it on a laptop and seeing how well it works. And it doesn't
really matter how reliably we can create minimal container images if
the workstation image doesn't compose or doesn't log in.

We need clear naming for the workstation releases we do. If we want to
call them F30, F30.1, F31, F31.1 we can, but that doesn't inherently
reduce load on releng or remove the need for infrastructure freezes
... my assumption was that the idea of delaying F31 is to actually
*not* do an operating system release for a year, and that's something
that I don't think we should do on an ongoing basis.

Owen

Post by Brendan Conoboy

Post by Josh Boyer

Post by Owen Taylor

Post by Josh Boyer

Post by Owen Taylor
But for the next thousand or so Fedora developers, the release cycle
is actually not a big deal - not something that takes much of their
time - and it gives them a regular place to land feature work. And
Fedora users appreciate a timely updated operating system (without
having random rebases trickle in.)

It's not a big deal because they don't participate in making the
release nor do they really know about all the duct tape and bailing
wire holding all the machinery in place. Just like most of them don't
appreciate the heroics involved from Fedora QA and rel-eng and the
dozen or so people that regularly go well beyond the extra mile to get
it out the door. This isn't done out of malice on their part. It's
mostly that they just don't see it.

If we distribute this work out across the 1000 developers, we'll make
life better for heroes, and it's *still* not going to be a big deal
for the 1000 developers.

I would say it will be a learning curve and there will be complaints
because it's change and people seem to only want change if it's the
change they want or otherwise doesn't impact them. That's going to be
a big deal to some :) Generally though, yes. We need to distribute
the burden to the package sets and whomever maintains them.

Post by Owen Taylor

Post by Josh Boyer

Post by Owen Taylor
In other words, the "technical debt" we are trying to solve here is
not project wide and doesn't justify slowing down the whole project
permanently.

I completely disagree. Our release process and tooling is built on
heroism and tech debt. At some point, that is going to cause
significant burnout and then people will just wonder what happened
when those people stop doing releases. I think it's better to raise
awareness at the project level, and build something that is
sustainable rather than predicated on "small group of people killing
themselves for the greater good". That starts with individual package
CI gating, and goes all the way through integration testing between
packages in a gated manner, into how we actually *create* all of that
plus the things we deem worth of releasing.

I think you are misunderstanding me somehow. I'm 100% on-board with
improving the processes, catching compose problems in an automated and
early fashion, making developers fix their own problems, and generally
making releases a less heroic process. And if that requires a
temporary cadence chance to get the retooling done, then it's worth
it.

Ah, good!

Post by Owen Taylor
But Brendan's proposal to permanently slow down the cadence seems to
make the assumption that the current release cycle is a heroic effort
for the entire project - that we're moving too fast to stay on our
feet. And I don't think that's the case.

That would be a concern to me as well, assuming we stuck with a
*single* cadence and a *single* platform. With the lifecycle
Objective, aren't we targeting multiple cadences and lifecycles? I
mean, we have this today with Fedora Atomic/CoreOS vs. "regular"
Fedora. Maybe Brendan was thinking singular, but I would very much
like the ability in our tooling a process to have both the faster
moving cadence of today AND a slower moving one. That's another
reason I view a temporary halt as worthwhile. If done correctly, it
enables Fedora to fill more usecases and lifecycles than we could ever
accomplish with a single release.

Josh has it: I am indeed thinking about how this works with the
lifecycle objective. When we think about things as all or nothing we
rapidly get into a zero sum game: Let's make this part of Fedora worse
to make this other part better. We don't have to do that, we can just
make Fedora better. That's what having multiple cadences and
lifecycles is all about. So it is important to talk about how the
rules would change if F30 takes a year, because if the rules don't
change it might diminish one of the great things about Fedora.
--
Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc.
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.o

[Paul Frields]

Post by Owen Taylor
We can definitely talk about whether moving to a slower cadence for
certain parts of the base platform. But people don't judge Fedora on
how beautifully we maintain glibc and gcc - they mostly judge it by
installing it on a laptop and seeing how well it works. And it doesn't
really matter how reliably we can create minimal container images if
the workstation image doesn't compose or doesn't log in.
We need clear naming for the workstation releases we do. If we want to
call them F30, F30.1, F31, F31.1 we can, but that doesn't inherently
reduce load on releng or remove the need for infrastructure freezes
... my assumption was that the idea of delaying F31 is to actually
*not* do an operating system release for a year, and that's something
that I don't think we should do on an ongoing basis.

Heard, understood, and agreed. Shortening the compose is a root
problem to fix if we want to be able to judge reliability of what we
want to release. That way we can test it more or less constantly.

As for freezes... right now we freeze for a total of roughly 2-3
months out of 12. We can avoid that with a pause -- at least in part,
and maybe in total if we finish enough work to minimize freeze coming
out of it. That's why taking the break makes sense.

Reclaiming that time, year after year, should be a knock-on effect of
being able to release more frequently as a non-event. Then there's
less (maybe no) reason to freeze -- the worst thing that happens is we
pull the lever the next day. So a healthy part of our investment
should be in minimizing recovery time.

--
Paul
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedoraproject.or

[Japheth Cleaver]

Post by Owen Taylor
We can definitely talk about whether moving to a slower cadence for
certain parts of the base platform. But people don't judge Fedora on
how beautifully we maintain glibc and gcc - they mostly judge it by
installing it on a laptop and seeing how well it works. And it doesn't
really matter how reliably we can create minimal container images if
the workstation image doesn't compose or doesn't log in.

I'm not sure that that's fair. I mean, I judge Fedora on how much its
features are likely to cause problems 9-36 months down the road in an EL
release... or now, in the case of cross-distro integration and support.
I could care less about GNOME, but care very much about gcc, glibc, the
kernel, and packaging of libraries.

*Laptop users* certainly judge Fedora on how well it works on laptops,
and Workstation environment users judge Fedora on the Workstation spin.

Any other statement would pre-suppose the exact answers this and related
discussion would seem to be working towards.

-jc
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject

[Brendan Conoboy]

Post by Owen Taylor
We can definitely talk about whether moving to a slower cadence for
certain parts of the base platform. But people don't judge Fedora on
how beautifully we maintain glibc and gcc - they mostly judge it by
installing it on a laptop and seeing how well it works. And it doesn't
really matter how reliably we can create minimal container images if
the workstation image doesn't compose or doesn't log in.

It's true that the #1 use case for Fedora is the desktop, but that
isn't all Fedora is used for. If Fedora is going to continue to grow
it needs to serve more uses. At the same time it needs to keep doing
a great job as a desktop- so we have to look at solutions that promote
both.

How beautifully we maintain platform pieces like gcc, glibc, and other
commonly used shared libraries is actually tremendously important to
many users, to many developers, because it affects their ability to
adopt Fedora as a base to do the things they want to do.

Post by Owen Taylor
We need clear naming for the workstation releases we do. If we want to
call them F30, F30.1, F31, F31.1 we can, but that doesn't inherently
reduce load on releng or remove the need for infrastructure freezes
... my assumption was that the idea of delaying F31 is to actually
*not* do an operating system release for a year, and that's something
that I don't think we should do on an ongoing basis.

Paul's proposal was definitely a one-time pause for the reasons you
state. He requested we follow-up with additional questions and
suggestions so I'm questioning and suggesting taking it a step
further. We talk about rolling releases, but people are skeptical due
to rawhide instability. What does it look like if the "rolling"
happens on top of an otherwise stable platform where fundamentals like
compilers, libraries and core system tools are held steady, but things
on top move fast? Maybe you don't need an F30.1, maybe it means F30
just keeps getting nice incremental updates for as long as the
editions want to stick with it. Variable lifecycle or cadence can
open up these kinds of options. Some things are better fast. Some
things are better slow.

Post by Owen Taylor
Owen

Post by Brendan Conoboy

Post by Josh Boyer

Post by Owen Taylor

Post by Josh Boyer

Post by Owen Taylor
But for the next thousand or so Fedora developers, the release cycle
is actually not a big deal - not something that takes much of their
time - and it gives them a regular place to land feature work. And
Fedora users appreciate a timely updated operating system (without
having random rebases trickle in.)

It's not a big deal because they don't participate in making the
release nor do they really know about all the duct tape and bailing
wire holding all the machinery in place. Just like most of them don't
appreciate the heroics involved from Fedora QA and rel-eng and the
dozen or so people that regularly go well beyond the extra mile to get
it out the door. This isn't done out of malice on their part. It's
mostly that they just don't see it.

If we distribute this work out across the 1000 developers, we'll make
life better for heroes, and it's *still* not going to be a big deal
for the 1000 developers.

I would say it will be a learning curve and there will be complaints
because it's change and people seem to only want change if it's the
change they want or otherwise doesn't impact them. That's going to be
a big deal to some :) Generally though, yes. We need to distribute
the burden to the package sets and whomever maintains them.

Post by Owen Taylor

Post by Josh Boyer

Post by Owen Taylor
In other words, the "technical debt" we are trying to solve here is
not project wide and doesn't justify slowing down the whole project
permanently.

I completely disagree. Our release process and tooling is built on
heroism and tech debt. At some point, that is going to cause
significant burnout and then people will just wonder what happened
when those people stop doing releases. I think it's better to raise
awareness at the project level, and build something that is
sustainable rather than predicated on "small group of people killing
themselves for the greater good". That starts with individual package
CI gating, and goes all the way through integration testing between
packages in a gated manner, into how we actually *create* all of that
plus the things we deem worth of releasing.

I think you are misunderstanding me somehow. I'm 100% on-board with
improving the processes, catching compose problems in an automated and
early fashion, making developers fix their own problems, and generally
making releases a less heroic process. And if that requires a
temporary cadence chance to get the retooling done, then it's worth
it.

Ah, good!

Post by Owen Taylor
But Brendan's proposal to permanently slow down the cadence seems to
make the assumption that the current release cycle is a heroic effort
for the entire project - that we're moving too fast to stay on our
feet. And I don't think that's the case.

That would be a concern to me as well, assuming we stuck with a
*single* cadence and a *single* platform. With the lifecycle
Objective, aren't we targeting multiple cadences and lifecycles? I
mean, we have this today with Fedora Atomic/CoreOS vs. "regular"
Fedora. Maybe Brendan was thinking singular, but I would very much
like the ability in our tooling a process to have both the faster
moving cadence of today AND a slower moving one. That's another
reason I view a temporary halt as worthwhile. If done correctly, it
enables Fedora to fill more usecases and lifecycles than we could ever
accomplish with a single release.

Josh has it: I am indeed thinking about how this works with the
lifecycle objective. When we think about things as all or nothing we
rapidly get into a zero sum game: Let's make this part of Fedora worse
to make this other part better. We don't have to do that, we can just
make Fedora better. That's what having multiple cadences and
lifecycles is all about. So it is important to talk about how the
rules would change if F30 takes a year, because if the rules don't
change it might diminish one of the great things about Fedora.
--
Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc.
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

--
Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorap

[Brian (bex) Exelbierd]

Post by Brendan Conoboy
Paul's proposal was definitely a one-time pause for the reasons you
state. He requested we follow-up with additional questions and
suggestions so I'm questioning and suggesting taking it a step
further. We talk about rolling releases, but people are skeptical due
to rawhide instability. What does it look like if the "rolling"
happens on top of an otherwise stable platform where fundamentals like
compilers, libraries and core system tools are held steady, but things
on top move fast? Maybe you don't need an F30.1, maybe it means F30
just keeps getting nice incremental updates for as long as the
editions want to stick with it. Variable lifecycle or cadence can
open up these kinds of options. Some things are better fast. Some
things are better slow.

This. Yes This. +100

I think that Fedora's role as an innovater in the OS space means we
should be aggressively exploring this. Rolling Releases, Tech-Driven
Releases and Time-Based Releases all have well known positives and
negatives. All of the work that has been done on Modularity,
containers, flatpaks, OSTree, and more, gives us the opportunity to
really re-think this. While it is true there are dozens (or more)
additional solutions to the too-fast/too-slow and the
incompatible-libraries problems, these technologies seem to be gaining
the most adoption across a variety of use cases. They are all also
generally well supported in Fedora and we need to aggressively push
them to achieve this goal or determine where the dead-end is so we can
move to the next innovation.

I personal am hugely in favor of us adopting a bootable-base model
that allows us to iterate the kernel and the various user-space pieces
at the speed of the upstream, the user's desires and the builder's
desires[^0] all at the same time. While this will require us to do
some level of NxM matrix building and testing, a base that didn't have
to change often for most use cases reduces the matrix considerably.

I'd push Brendans' concept further and suggest that we try to
eliminate as many of the compilers, libraries and core system tools as
possible from this bootable-base so that those can iterate at their
own speed, perhaps 4 year for a laptop vendor and 30 day for a
experimental ARM device. Fedora as a project might not build output
for the whole range, but a build system that allowed us to help others
be successful would be a huge help here.

I recognize that I, like most people, see the world through the lens
of my specific use case, but remember, "Fedora creates an innovative
platform for hardware, clouds, and containers that enables software
developers and community members to build tailored solutions for their
users." As long as we don't block a use cases arbitrarily and we
leave room for that innovation and service we are doing the right
thing. The debate about what use cases should be done fully by
Fedora, enabled for a SIG/WG via our build system or done externally
by those using only the parts that make sense for them is a separate
debate.

regards,

bex

0: Builder's desires are the desires of the person who put the entire
system together to fulfill the needs of their community per our
mission statement. If my mythical llama herders need the oldest Libre
Office possible but the newest Rust packaging and whatever random
version of httpd that Fedora deems "stable", then that is what I
desire, even if the upstream or other non-llama herding users desire
something different. However, I'd also push that we should try to
reach a point where if a llama herder for non-llama reasons needs a
different httpd, they can just enable and use it (using the language
of modularity). In case it isn't clear, the "builder's desires"
includes the goals of every current lab, spin, and edition, separately
and where appropriate together.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorapr

[Peter Robinson]

On Wed, Nov 28, 2018 at 8:07 AM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd

Post by Brendan Conoboy
Paul's proposal was definitely a one-time pause for the reasons you
state. He requested we follow-up with additional questions and
suggestions so I'm questioning and suggesting taking it a step
further. We talk about rolling releases, but people are skeptical due
to rawhide instability. What does it look like if the "rolling"
happens on top of an otherwise stable platform where fundamentals like
compilers, libraries and core system tools are held steady, but things
on top move fast? Maybe you don't need an F30.1, maybe it means F30
just keeps getting nice incremental updates for as long as the
editions want to stick with it. Variable lifecycle or cadence can
open up these kinds of options. Some things are better fast. Some
things are better slow.

This. Yes This. +100
I think that Fedora's role as an innovater in the OS space means we
should be aggressively exploring this. Rolling Releases, Tech-Driven
Releases and Time-Based Releases all have well known positives and
negatives. All of the work that has been done on Modularity,
containers, flatpaks, OSTree, and more, gives us the opportunity to
really re-think this. While it is true there are dozens (or more)
additional solutions to the too-fast/too-slow and the
incompatible-libraries problems, these technologies seem to be gaining
the most adoption across a variety of use cases. They are all also
generally well supported in Fedora and we need to aggressively push
them to achieve this goal or determine where the dead-end is so we can
move to the next innovation.
I personal am hugely in favor of us adopting a bootable-base model
that allows us to iterate the kernel and the various user-space pieces
at the speed of the upstream, the user's desires and the builder's
desires[^0] all at the same time. While this will require us to do
some level of NxM matrix building and testing, a base that didn't have
to change often for most use cases reduces the matrix considerably.

The above doesn't make sense, you're saying "move as fast as upstream"
and "a base that doesn't change" in the same context, which is it?

Post by Brian (bex) Exelbierd
I'd push Brendans' concept further and suggest that we try to
eliminate as many of the compilers, libraries and core system tools as
possible from this bootable-base so that those can iterate at their
own speed, perhaps 4 year for a laptop vendor and 30 day for a
experimental ARM device. Fedora as a project might not build output
for the whole range, but a build system that allowed us to help others
be successful would be a huge help here.

Again what do you even mean by eliminate the compilers? Also how do we
not change something core, such as a compiler, for 4 years while also
change it every 30 days?

Post by Brian (bex) Exelbierd
I recognize that I, like most people, see the world through the lens
of my specific use case, but remember, "Fedora creates an innovative
platform for hardware, clouds, and containers that enables software
developers and community members to build tailored solutions for their
users." As long as we don't block a use cases arbitrarily and we
leave room for that innovation and service we are doing the right
thing. The debate about what use cases should be done fully by
Fedora, enabled for a SIG/WG via our build system or done externally
by those using only the parts that make sense for them is a separate
debate.

I agree on some of your points above, but TBH most of it reads like
some form of marketing coolaid!

Also it does account at all for any and/or all of the resources that
we'd need to even enact some of this, even if it's possible?

Post by Brian (bex) Exelbierd
0: Builder's desires are the desires of the person who put the entire
system together to fulfill the needs of their community per our
mission statement. If my mythical llama herders need the oldest Libre
Office possible but the newest Rust packaging and whatever random
version of httpd that Fedora deems "stable", then that is what I
desire, even if the upstream or other non-llama herding users desire
something different. However, I'd also push that we should try to
reach a point where if a llama herder for non-llama reasons needs a
different httpd, they can just enable and use it (using the language
of modularity). In case it isn't clear, the "builder's desires"
includes the goals of every current lab, spin, and edition, separately
and where appropriate together.
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@list

[Brian (bex) Exelbierd]

Post by Peter Robinson
On Wed, Nov 28, 2018 at 8:07 AM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd

Post by Brendan Conoboy
Paul's proposal was definitely a one-time pause for the reasons you
state. He requested we follow-up with additional questions and
suggestions so I'm questioning and suggesting taking it a step
further. We talk about rolling releases, but people are skeptical due
to rawhide instability. What does it look like if the "rolling"
happens on top of an otherwise stable platform where fundamentals like
compilers, libraries and core system tools are held steady, but things
on top move fast? Maybe you don't need an F30.1, maybe it means F30
just keeps getting nice incremental updates for as long as the
editions want to stick with it. Variable lifecycle or cadence can
open up these kinds of options. Some things are better fast. Some
things are better slow.

This. Yes This. +100
I think that Fedora's role as an innovater in the OS space means we
should be aggressively exploring this. Rolling Releases, Tech-Driven
Releases and Time-Based Releases all have well known positives and
negatives. All of the work that has been done on Modularity,
containers, flatpaks, OSTree, and more, gives us the opportunity to
really re-think this. While it is true there are dozens (or more)
additional solutions to the too-fast/too-slow and the
incompatible-libraries problems, these technologies seem to be gaining
the most adoption across a variety of use cases. They are all also
generally well supported in Fedora and we need to aggressively push
them to achieve this goal or determine where the dead-end is so we can
move to the next innovation.
I personal am hugely in favor of us adopting a bootable-base model
that allows us to iterate the kernel and the various user-space pieces
at the speed of the upstream, the user's desires and the builder's
desires[^0] all at the same time. While this will require us to do
some level of NxM matrix building and testing, a base that didn't have
to change often for most use cases reduces the matrix considerably.

The above doesn't make sense, you're saying "move as fast as upstream"
and "a base that doesn't change" in the same context, which is it?

I've failed to be clear - sorry about that. Let me try again.

Please remember that I tend think from the lens of user space, not
kernel space. So there may be detail errors in this, I am hoping the
concepts are valid though.

In general, I can run various versions of my applications against
multiple different kernels, for example. Therefore, if I have a
kernel that changed once a year, it isn't going to, for many
applications, stop me from changing versions multiple times during the
year. Therefore if Fedora had a stabilized bootable base, I could
move my applications at the cadence of upstream, or a stabilized
release (not at all) or at the speed in the middle I want. Fedora
might not build the entire range of that, but I am not prevented from
choosing amongst multiple Fedora provided options (stable vs devel or
all supported upstream releases, for example).

The bootable base would change based on Fedora's needs. Perhaps we
decide want to new kernels (again sorry for my failings in this field)
every 6 months to introduce new drivers and hardware support. Someone
who wants faster can self-build for their community/needs more
frequently or a hardware vendor might want a kernel that doesn't
change as often and is backported. Fedora may not build the whole
range here either.

Post by Peter Robinson

Post by Brian (bex) Exelbierd
I'd push Brendans' concept further and suggest that we try to
eliminate as many of the compilers, libraries and core system tools as
possible from this bootable-base so that those can iterate at their
own speed, perhaps 4 year for a laptop vendor and 30 day for a
experimental ARM device. Fedora as a project might not build output
for the whole range, but a build system that allowed us to help others
be successful would be a huge help here.

Again what do you even mean by eliminate the compilers? Also how do we
not change something core, such as a compiler, for 4 years while also
change it every 30 days?

"Eliminate the compilers" was meant to mean, make them modules or
non-bootable base components as much as possible. I realize that for
something like glibc that can be hard to impossible and for things
like Fortran a non-issue.

Communities have different needs, and every time we freeze or force
change we create challenges for those needs to be met. My point is
that by keeping our base to the "light up the machine level (another
hated idiom) and get containers/flatpaks/etc running" we can allow the
builder to focus on their community's needs or the user to get their
own too-fast/too-slow balance.

What I'd like to forego is the long "what is base" conversation. I'd
pull it back to what does it take to boot the machine and get
containers/flatpaks/modules running. Everything after that should be
flexible, even if we put down requirements and rules for the use cases
we want to tackle and the pieces we build and deliver.

Post by Peter Robinson

Post by Brian (bex) Exelbierd
I recognize that I, like most people, see the world through the lens
of my specific use case, but remember, "Fedora creates an innovative
platform for hardware, clouds, and containers that enables software
developers and community members to build tailored solutions for their
users." As long as we don't block a use cases arbitrarily and we
leave room for that innovation and service we are doing the right
thing. The debate about what use cases should be done fully by
Fedora, enabled for a SIG/WG via our build system or done externally
by those using only the parts that make sense for them is a separate
debate.

I agree on some of your points above, but TBH most of it reads like
some form of marketing coolaid!

Marketing Koolaid (in the interesting and love sense, not the
pejorative) would be really good for Fedora right now, if we want
increased adoption and contribution. Flexibility to allow our mission
statement to keep being true is critical. I fully admit that my
skills are not in distribution building and that you and others in
this thread have those skills in greater capacity than I do. But I
don't hear anyone starting from the premise of our mission statement
and then moving forward. I feel like a build/distribution focused on
easing the build process and ability to provide across the full
too-fast/too-slow spectrum, even if we as a project don't fill the
whole spectrum, is crucial to Fedora's ongoing success.

regards,

bex

Post by Peter Robinson
Also it does account at all for any and/or all of the resources that
we'd need to even enact some of this, even if it's possible?

Post by Brian (bex) Exelbierd
0: Builder's desires are the desires of the person who put the entire
system together to fulfill the needs of their community per our
mission statement. If my mythical llama herders need the oldest Libre
Office possible but the newest Rust packaging and whatever random
version of httpd that Fedora deems "stable", then that is what I
desire, even if the upstream or other non-llama herding users desire
something different. However, I'd also push that we should try to
reach a point where if a llama herder for non-llama reasons needs a
different httpd, they can just enable and use it (using the language
of modularity). In case it isn't clear, the "builder's desires"
includes the goals of every current lab, spin, and edition, separately
and where appropriate together.
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

--
Brian (bex) Exelbierd | ***@redhat.com | ***@pobox.com
Fedora Community Action & Impact Coordinator
@bexelbie | http://www.winglemeyer.org
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedorapro

[Igor Gnatenko]

On Wed, Nov 28, 2018 at 1:14 PM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd

Post by Peter Robinson
On Wed, Nov 28, 2018 at 8:07 AM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd

Post by Brendan Conoboy
Paul's proposal was definitely a one-time pause for the reasons you
state. He requested we follow-up with additional questions and
suggestions so I'm questioning and suggesting taking it a step
further. We talk about rolling releases, but people are skeptical due
to rawhide instability. What does it look like if the "rolling"
happens on top of an otherwise stable platform where fundamentals like
compilers, libraries and core system tools are held steady, but things
on top move fast? Maybe you don't need an F30.1, maybe it means F30
just keeps getting nice incremental updates for as long as the
editions want to stick with it. Variable lifecycle or cadence can
open up these kinds of options. Some things are better fast. Some
things are better slow.

This. Yes This. +100
I think that Fedora's role as an innovater in the OS space means we
should be aggressively exploring this. Rolling Releases, Tech-Driven
Releases and Time-Based Releases all have well known positives and
negatives. All of the work that has been done on Modularity,
containers, flatpaks, OSTree, and more, gives us the opportunity to
really re-think this. While it is true there are dozens (or more)
additional solutions to the too-fast/too-slow and the
incompatible-libraries problems, these technologies seem to be gaining
the most adoption across a variety of use cases. They are all also
generally well supported in Fedora and we need to aggressively push
them to achieve this goal or determine where the dead-end is so we can
move to the next innovation.
I personal am hugely in favor of us adopting a bootable-base model
that allows us to iterate the kernel and the various user-space pieces
at the speed of the upstream, the user's desires and the builder's
desires[^0] all at the same time. While this will require us to do
some level of NxM matrix building and testing, a base that didn't have
to change often for most use cases reduces the matrix considerably.

The above doesn't make sense, you're saying "move as fast as upstream"
and "a base that doesn't change" in the same context, which is it?

I've failed to be clear - sorry about that. Let me try again.
Please remember that I tend think from the lens of user space, not
kernel space. So there may be detail errors in this, I am hoping the
concepts are valid though.
In general, I can run various versions of my applications against
multiple different kernels, for example. Therefore, if I have a
kernel that changed once a year, it isn't going to, for many
applications, stop me from changing versions multiple times during the
year. Therefore if Fedora had a stabilized bootable base, I could
move my applications at the cadence of upstream, or a stabilized
release (not at all) or at the speed in the middle I want. Fedora
might not build the entire range of that, but I am not prevented from
choosing amongst multiple Fedora provided options (stable vs devel or
all supported upstream releases, for example).

So you mean that you want CentOS stable base and latest software you want?

Post by Brian (bex) Exelbierd
The bootable base would change based on Fedora's needs. Perhaps we
decide want to new kernels (again sorry for my failings in this field)
every 6 months to introduce new drivers and hardware support. Someone
who wants faster can self-build for their community/needs more
frequently or a hardware vendor might want a kernel that doesn't
change as often and is backported. Fedora may not build the whole
range here either.

Post by Peter Robinson

Post by Brian (bex) Exelbierd
I'd push Brendans' concept further and suggest that we try to
eliminate as many of the compilers, libraries and core system tools as
possible from this bootable-base so that those can iterate at their
own speed, perhaps 4 year for a laptop vendor and 30 day for a
experimental ARM device. Fedora as a project might not build output
for the whole range, but a build system that allowed us to help others
be successful would be a huge help here.

Again what do you even mean by eliminate the compilers? Also how do we
not change something core, such as a compiler, for 4 years while also
change it every 30 days?

"Eliminate the compilers" was meant to mean, make them modules or
non-bootable base components as much as possible. I realize that for
something like glibc that can be hard to impossible and for things
like Fortran a non-issue.
Communities have different needs, and every time we freeze or force
change we create challenges for those needs to be met. My point is
that by keeping our base to the "light up the machine level (another
hated idiom) and get containers/flatpaks/etc running" we can allow the
builder to focus on their community's needs or the user to get their
own too-fast/too-slow balance.
What I'd like to forego is the long "what is base" conversation. I'd
pull it back to what does it take to boot the machine and get
containers/flatpaks/modules running. Everything after that should be
flexible, even if we put down requirements and rules for the use cases
we want to tackle and the pieces we build and deliver.

Post by Peter Robinson

Post by Brian (bex) Exelbierd
I recognize that I, like most people, see the world through the lens
of my specific use case, but remember, "Fedora creates an innovative
platform for hardware, clouds, and containers that enables software
developers and community members to build tailored solutions for their
users." As long as we don't block a use cases arbitrarily and we
leave room for that innovation and service we are doing the right
thing. The debate about what use cases should be done fully by
Fedora, enabled for a SIG/WG via our build system or done externally
by those using only the parts that make sense for them is a separate
debate.

I agree on some of your points above, but TBH most of it reads like
some form of marketing coolaid!

Marketing Koolaid (in the interesting and love sense, not the
pejorative) would be really good for Fedora right now, if we want
increased adoption and contribution. Flexibility to allow our mission
statement to keep being true is critical. I fully admit that my
skills are not in distribution building and that you and others in
this thread have those skills in greater capacity than I do. But I
don't hear anyone starting from the premise of our mission statement
and then moving forward. I feel like a build/distribution focused on
easing the build process and ability to provide across the full
too-fast/too-slow spectrum, even if we as a project don't fill the
whole spectrum, is crucial to Fedora's ongoing success.

I have nothing against, but stopping release for some mysterious
reasons seems bad idea, but if we get some specific goals which we try
to achieve -- I might even support this idea.

Post by Brian (bex) Exelbierd
regards,
bex

Post by Peter Robinson
Also it does account at all for any and/or all of the resources that
we'd need to even enact some of this, even if it's possible?

Post by Brian (bex) Exelbierd
0: Builder's desires are the desires of the person who put the entire
system together to fulfill the needs of their community per our
mission statement. If my mythical llama herders need the oldest Libre
Office possible but the newest Rust packaging and whatever random
version of httpd that Fedora deems "stable", then that is what I
desire, even if the upstream or other non-llama herding users desire
something different. However, I'd also push that we should try to
reach a point where if a llama herder for non-llama reasons needs a
different httpd, they can just enable and use it (using the language
of modularity). In case it isn't clear, the "builder's desires"
includes the goals of every current lab, spin, and edition, separately
and where appropriate together.
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

--
Fedora Community Action & Impact Coordinator
@bexelbie | http://www.winglemeyer.org
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedoraproject.

[Brian (bex) Exelbierd]

On Wed, Nov 28, 2018 at 1:31 PM Igor Gnatenko

Post by Igor Gnatenko
On Wed, Nov 28, 2018 at 1:14 PM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd

Post by Peter Robinson
On Wed, Nov 28, 2018 at 8:07 AM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd

Post by Brendan Conoboy
Paul's proposal was definitely a one-time pause for the reasons you
state. He requested we follow-up with additional questions and
suggestions so I'm questioning and suggesting taking it a step
further. We talk about rolling releases, but people are skeptical due
to rawhide instability. What does it look like if the "rolling"
happens on top of an otherwise stable platform where fundamentals like
compilers, libraries and core system tools are held steady, but things
on top move fast? Maybe you don't need an F30.1, maybe it means F30
just keeps getting nice incremental updates for as long as the
editions want to stick with it. Variable lifecycle or cadence can
open up these kinds of options. Some things are better fast. Some
things are better slow.

This. Yes This. +100
I think that Fedora's role as an innovater in the OS space means we
should be aggressively exploring this. Rolling Releases, Tech-Driven
Releases and Time-Based Releases all have well known positives and
negatives. All of the work that has been done on Modularity,
containers, flatpaks, OSTree, and more, gives us the opportunity to
really re-think this. While it is true there are dozens (or more)
additional solutions to the too-fast/too-slow and the
incompatible-libraries problems, these technologies seem to be gaining
the most adoption across a variety of use cases. They are all also
generally well supported in Fedora and we need to aggressively push
them to achieve this goal or determine where the dead-end is so we can
move to the next innovation.
I personal am hugely in favor of us adopting a bootable-base model
that allows us to iterate the kernel and the various user-space pieces
at the speed of the upstream, the user's desires and the builder's
desires[^0] all at the same time. While this will require us to do
some level of NxM matrix building and testing, a base that didn't have
to change often for most use cases reduces the matrix considerably.

The above doesn't make sense, you're saying "move as fast as upstream"
and "a base that doesn't change" in the same context, which is it?

I've failed to be clear - sorry about that. Let me try again.
Please remember that I tend think from the lens of user space, not
kernel space. So there may be detail errors in this, I am hoping the
concepts are valid though.
In general, I can run various versions of my applications against
multiple different kernels, for example. Therefore, if I have a
kernel that changed once a year, it isn't going to, for many
applications, stop me from changing versions multiple times during the
year. Therefore if Fedora had a stabilized bootable base, I could
move my applications at the cadence of upstream, or a stabilized
release (not at all) or at the speed in the middle I want. Fedora
might not build the entire range of that, but I am not prevented from
choosing amongst multiple Fedora provided options (stable vs devel or
all supported upstream releases, for example).

So you mean that you want CentOS stable base and latest software you want?

Nope. I want CentOS to serve their user base. If the CentOS project
chooses to formally become part of Fedora and to release a
longer-maintained base to their users under the banner of our mission,
I welcome them with open arms. But, that is their decision.

I don't think Fedora wants to get into 10 year life cycles and that
level of backporting. But, accepting a new kernel/bootable base
shouldn't force me to take a new version of LibreOffice, for example.

Post by Igor Gnatenko

Post by Brian (bex) Exelbierd
The bootable base would change based on Fedora's needs. Perhaps we
decide want to new kernels (again sorry for my failings in this field)
every 6 months to introduce new drivers and hardware support. Someone
who wants faster can self-build for their community/needs more
frequently or a hardware vendor might want a kernel that doesn't
change as often and is backported. Fedora may not build the whole
range here either.

Post by Peter Robinson

Post by Brian (bex) Exelbierd
I'd push Brendans' concept further and suggest that we try to
eliminate as many of the compilers, libraries and core system tools as
possible from this bootable-base so that those can iterate at their
own speed, perhaps 4 year for a laptop vendor and 30 day for a
experimental ARM device. Fedora as a project might not build output
for the whole range, but a build system that allowed us to help others
be successful would be a huge help here.

Again what do you even mean by eliminate the compilers? Also how do we
not change something core, such as a compiler, for 4 years while also
change it every 30 days?

"Eliminate the compilers" was meant to mean, make them modules or
non-bootable base components as much as possible. I realize that for
something like glibc that can be hard to impossible and for things
like Fortran a non-issue.
Communities have different needs, and every time we freeze or force
change we create challenges for those needs to be met. My point is
that by keeping our base to the "light up the machine level (another
hated idiom) and get containers/flatpaks/etc running" we can allow the
builder to focus on their community's needs or the user to get their
own too-fast/too-slow balance.
What I'd like to forego is the long "what is base" conversation. I'd
pull it back to what does it take to boot the machine and get
containers/flatpaks/modules running. Everything after that should be
flexible, even if we put down requirements and rules for the use cases
we want to tackle and the pieces we build and deliver.

Post by Peter Robinson

Post by Brian (bex) Exelbierd
I recognize that I, like most people, see the world through the lens
of my specific use case, but remember, "Fedora creates an innovative
platform for hardware, clouds, and containers that enables software
developers and community members to build tailored solutions for their
users." As long as we don't block a use cases arbitrarily and we
leave room for that innovation and service we are doing the right
thing. The debate about what use cases should be done fully by
Fedora, enabled for a SIG/WG via our build system or done externally
by those using only the parts that make sense for them is a separate
debate.

I agree on some of your points above, but TBH most of it reads like
some form of marketing coolaid!

Marketing Koolaid (in the interesting and love sense, not the
pejorative) would be really good for Fedora right now, if we want
increased adoption and contribution. Flexibility to allow our mission
statement to keep being true is critical. I fully admit that my
skills are not in distribution building and that you and others in
this thread have those skills in greater capacity than I do. But I
don't hear anyone starting from the premise of our mission statement
and then moving forward. I feel like a build/distribution focused on
easing the build process and ability to provide across the full
too-fast/too-slow spectrum, even if we as a project don't fill the
whole spectrum, is crucial to Fedora's ongoing success.

I have nothing against, but stopping release for some mysterious
reasons seems bad idea, but if we get some specific goals which we try
to achieve -- I might even support this idea.

I am not sure releases as a construct, are as useful in their current
form to us today as they were in the past. I (emphatically) do NOT
want us to stop producing and releasing new software. In fact, I want
it released faster and more often, but without the accompanying user
disruption of forced upgrade/rolling releases. I want our
editions/spins/labs/remixes/etc. to be able to assemble the Fedora
that meet's their communities needs from the amazing pool of
contributions the project as a whole has.

Fedora may not build or even endorse everything that is made, but this
allows our mission statement to be put into practice. It allows
Fedora to become the easiest distribution to base your solution on.

regards,

bex

Post by Igor Gnatenko

Post by Brian (bex) Exelbierd
regards,
bex

Post by Peter Robinson
Also it does account at all for any and/or all of the resources that
we'd need to even enact some of this, even if it's possible?

Post by Brian (bex) Exelbierd
0: Builder's desires are the desires of the person who put the entire
system together to fulfill the needs of their community per our
mission statement. If my mythical llama herders need the oldest Libre
Office possible but the newest Rust packaging and whatever random
version of httpd that Fedora deems "stable", then that is what I
desire, even if the upstream or other non-llama herding users desire
something different. However, I'd also push that we should try to
reach a point where if a llama herder for non-llama reasons needs a
different httpd, they can just enable and use it (using the language
of modularity). In case it isn't clear, the "builder's desires"
includes the goals of every current lab, spin, and edition, separately
and where appropriate together.
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

--
Fedora Community Action & Impact Coordinator
@bexelbie | http://www.winglemeyer.org
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

--
Brian (bex) Exelbierd | ***@redhat.com | ***@pobox.com
Fedora Community Action & Impact Coordinator
@bexelbie | http://www.winglemeyer.org
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@l

[Igor Gnatenko]

On Wed, Nov 28, 2018 at 1:49 PM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd
On Wed, Nov 28, 2018 at 1:31 PM Igor Gnatenko

Post by Igor Gnatenko
On Wed, Nov 28, 2018 at 1:14 PM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd

Post by Peter Robinson
On Wed, Nov 28, 2018 at 8:07 AM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd

Post by Brendan Conoboy
Paul's proposal was definitely a one-time pause for the reasons you
state. He requested we follow-up with additional questions and
suggestions so I'm questioning and suggesting taking it a step
further. We talk about rolling releases, but people are skeptical due
to rawhide instability. What does it look like if the "rolling"
happens on top of an otherwise stable platform where fundamentals like
compilers, libraries and core system tools are held steady, but things
on top move fast? Maybe you don't need an F30.1, maybe it means F30
just keeps getting nice incremental updates for as long as the
editions want to stick with it. Variable lifecycle or cadence can
open up these kinds of options. Some things are better fast. Some
things are better slow.

This. Yes This. +100
I think that Fedora's role as an innovater in the OS space means we
should be aggressively exploring this. Rolling Releases, Tech-Driven
Releases and Time-Based Releases all have well known positives and
negatives. All of the work that has been done on Modularity,
containers, flatpaks, OSTree, and more, gives us the opportunity to
really re-think this. While it is true there are dozens (or more)
additional solutions to the too-fast/too-slow and the
incompatible-libraries problems, these technologies seem to be gaining
the most adoption across a variety of use cases. They are all also
generally well supported in Fedora and we need to aggressively push
them to achieve this goal or determine where the dead-end is so we can
move to the next innovation.
I personal am hugely in favor of us adopting a bootable-base model
that allows us to iterate the kernel and the various user-space pieces
at the speed of the upstream, the user's desires and the builder's
desires[^0] all at the same time. While this will require us to do
some level of NxM matrix building and testing, a base that didn't have
to change often for most use cases reduces the matrix considerably.

The above doesn't make sense, you're saying "move as fast as upstream"
and "a base that doesn't change" in the same context, which is it?

I've failed to be clear - sorry about that. Let me try again.
Please remember that I tend think from the lens of user space, not
kernel space. So there may be detail errors in this, I am hoping the
concepts are valid though.
In general, I can run various versions of my applications against
multiple different kernels, for example. Therefore, if I have a
kernel that changed once a year, it isn't going to, for many
applications, stop me from changing versions multiple times during the
year. Therefore if Fedora had a stabilized bootable base, I could
move my applications at the cadence of upstream, or a stabilized
release (not at all) or at the speed in the middle I want. Fedora
might not build the entire range of that, but I am not prevented from
choosing amongst multiple Fedora provided options (stable vs devel or
all supported upstream releases, for example).

So you mean that you want CentOS stable base and latest software you want?

Nope. I want CentOS to serve their user base. If the CentOS project
chooses to formally become part of Fedora and to release a
longer-maintained base to their users under the banner of our mission,
I welcome them with open arms. But, that is their decision.
I don't think Fedora wants to get into 10 year life cycles and that
level of backporting. But, accepting a new kernel/bootable base
shouldn't force me to take a new version of LibreOffice, for example.

So what you are saying here is what "first" version of Modularity was
about. And it didn't work out because of many reasons. One of them is
no useful CI infrastructure (still not solved) which didn't allow
packagers to remove useless BuildRequires for tests (because rpmbuild
is still tte way to run test suite).

If we can get commitment from people who maintain "minimal viable
base" to move their tests to CI and get FTBFS fixes in time -- it
might work.

Post by Brian (bex) Exelbierd

Post by Igor Gnatenko

Post by Brian (bex) Exelbierd
The bootable base would change based on Fedora's needs. Perhaps we
decide want to new kernels (again sorry for my failings in this field)
every 6 months to introduce new drivers and hardware support. Someone
who wants faster can self-build for their community/needs more
frequently or a hardware vendor might want a kernel that doesn't
change as often and is backported. Fedora may not build the whole
range here either.

Post by Peter Robinson

Post by Brian (bex) Exelbierd
I'd push Brendans' concept further and suggest that we try to
eliminate as many of the compilers, libraries and core system tools as
possible from this bootable-base so that those can iterate at their
own speed, perhaps 4 year for a laptop vendor and 30 day for a
experimental ARM device. Fedora as a project might not build output
for the whole range, but a build system that allowed us to help others
be successful would be a huge help here.

Again what do you even mean by eliminate the compilers? Also how do we
not change something core, such as a compiler, for 4 years while also
change it every 30 days?

"Eliminate the compilers" was meant to mean, make them modules or
non-bootable base components as much as possible. I realize that for
something like glibc that can be hard to impossible and for things
like Fortran a non-issue.
Communities have different needs, and every time we freeze or force
change we create challenges for those needs to be met. My point is
that by keeping our base to the "light up the machine level (another
hated idiom) and get containers/flatpaks/etc running" we can allow the
builder to focus on their community's needs or the user to get their
own too-fast/too-slow balance.
What I'd like to forego is the long "what is base" conversation. I'd
pull it back to what does it take to boot the machine and get
containers/flatpaks/modules running. Everything after that should be
flexible, even if we put down requirements and rules for the use cases
we want to tackle and the pieces we build and deliver.

Post by Peter Robinson

Post by Brian (bex) Exelbierd
I recognize that I, like most people, see the world through the lens
of my specific use case, but remember, "Fedora creates an innovative
platform for hardware, clouds, and containers that enables software
developers and community members to build tailored solutions for their
users." As long as we don't block a use cases arbitrarily and we
leave room for that innovation and service we are doing the right
thing. The debate about what use cases should be done fully by
Fedora, enabled for a SIG/WG via our build system or done externally
by those using only the parts that make sense for them is a separate
debate.

I agree on some of your points above, but TBH most of it reads like
some form of marketing coolaid!

Marketing Koolaid (in the interesting and love sense, not the
pejorative) would be really good for Fedora right now, if we want
increased adoption and contribution. Flexibility to allow our mission
statement to keep being true is critical. I fully admit that my
skills are not in distribution building and that you and others in
this thread have those skills in greater capacity than I do. But I
don't hear anyone starting from the premise of our mission statement
and then moving forward. I feel like a build/distribution focused on
easing the build process and ability to provide across the full
too-fast/too-slow spectrum, even if we as a project don't fill the
whole spectrum, is crucial to Fedora's ongoing success.

I have nothing against, but stopping release for some mysterious
reasons seems bad idea, but if we get some specific goals which we try
to achieve -- I might even support this idea.

I am not sure releases as a construct, are as useful in their current
form to us today as they were in the past. I (emphatically) do NOT
want us to stop producing and releasing new software. In fact, I want
it released faster and more often, but without the accompanying user
disruption of forced upgrade/rolling releases. I want our
editions/spins/labs/remixes/etc. to be able to assemble the Fedora
that meet's their communities needs from the amazing pool of
contributions the project as a whole has.
Fedora may not build or even endorse everything that is made, but this
allows our mission statement to be put into practice. It allows
Fedora to become the easiest distribution to base your solution on.

Yes, I fully support this.

Post by Brian (bex) Exelbierd
regards,
bex

Post by Igor Gnatenko

Post by Brian (bex) Exelbierd
regards,
bex

Post by Peter Robinson
Also it does account at all for any and/or all of the resources that
we'd need to even enact some of this, even if it's possible?

Post by Brian (bex) Exelbierd
0: Builder's desires are the desires of the person who put the entire
system together to fulfill the needs of their community per our
mission statement. If my mythical llama herders need the oldest Libre
Office possible but the newest Rust packaging and whatever random
version of httpd that Fedora deems "stable", then that is what I
desire, even if the upstream or other non-llama herding users desire
something different. However, I'd also push that we should try to
reach a point where if a llama herder for non-llama reasons needs a
different httpd, they can just enable and use it (using the language
of modularity). In case it isn't clear, the "builder's desires"
includes the goals of every current lab, spin, and edition, separately
and where appropriate together.
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

--
Fedora Community Action & Impact Coordinator
@bexelbie | http://www.winglemeyer.org
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

--
Fedora Community Action & Impact Coordinator
@bexelbie | http://www.winglemeyer.org
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list

[Fabio Valentini]

On Wed, Nov 28, 2018 at 1:07 PM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd

Post by Peter Robinson
On Wed, Nov 28, 2018 at 8:07 AM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd

Post by Brendan Conoboy
Paul's proposal was definitely a one-time pause for the reasons you
state. He requested we follow-up with additional questions and
suggestions so I'm questioning and suggesting taking it a step
further. We talk about rolling releases, but people are skeptical due
to rawhide instability. What does it look like if the "rolling"
happens on top of an otherwise stable platform where fundamentals like
compilers, libraries and core system tools are held steady, but things
on top move fast? Maybe you don't need an F30.1, maybe it means F30
just keeps getting nice incremental updates for as long as the
editions want to stick with it. Variable lifecycle or cadence can
open up these kinds of options. Some things are better fast. Some
things are better slow.

This. Yes This. +100
I think that Fedora's role as an innovater in the OS space means we
should be aggressively exploring this. Rolling Releases, Tech-Driven
Releases and Time-Based Releases all have well known positives and
negatives. All of the work that has been done on Modularity,
containers, flatpaks, OSTree, and more, gives us the opportunity to
really re-think this. While it is true there are dozens (or more)
additional solutions to the too-fast/too-slow and the
incompatible-libraries problems, these technologies seem to be gaining
the most adoption across a variety of use cases. They are all also
generally well supported in Fedora and we need to aggressively push
them to achieve this goal or determine where the dead-end is so we can
move to the next innovation.
I personal am hugely in favor of us adopting a bootable-base model
that allows us to iterate the kernel and the various user-space pieces
at the speed of the upstream, the user's desires and the builder's
desires[^0] all at the same time. While this will require us to do
some level of NxM matrix building and testing, a base that didn't have
to change often for most use cases reduces the matrix considerably.

The above doesn't make sense, you're saying "move as fast as upstream"
and "a base that doesn't change" in the same context, which is it?

I've failed to be clear - sorry about that. Let me try again.
Please remember that I tend think from the lens of user space, not
kernel space. So there may be detail errors in this, I am hoping the
concepts are valid though.
In general, I can run various versions of my applications against
multiple different kernels, for example. Therefore, if I have a
kernel that changed once a year, it isn't going to, for many
applications, stop me from changing versions multiple times during the
year. Therefore if Fedora had a stabilized bootable base, I could
move my applications at the cadence of upstream, or a stabilized
release (not at all) or at the speed in the middle I want. Fedora
might not build the entire range of that, but I am not prevented from
choosing amongst multiple Fedora provided options (stable vs devel or
all supported upstream releases, for example).

I think the kernel is a bad example here. It's exceedly stable across
releases, so it's probably the one component that's least problematic
to upgrade during a fedora release. The fedora kernel team is already
doing that, and they are doing an excellent job, which they definitely
deserve more praise for than they get. For me, the frequent,
well-executed stable kernel updates are one thing that positively
distinguishes fedora from other non-rolling distros.

On the other hand, specific releases of GCC, glibc, (golang, ruby,
python, ocaml, perl, node, insert other compilers / interpreters here)
would be better examples of a "stable base". They really warrant "new
releases", since major updates of those components usually require
mass rebuilds of their dependent packages.

So, I'd rather draw the line between "base" and "rolling application
releases" *atop* the development environment that the versions of GCC,
glibc, python, golang, etc. make up, not below them. This is what many
packagers already do, from my experience. GNOME really is an exception
here, even KDE/KF5/Plasma is updated regularly during stable releases.

Whether the "base" system (that's used for development and building
all other packages atop it) is released every 6 months (aligning with
semi-annual releases of i.e. golang) or yearly (more in line with
~annual releases of GCC, python, etc.) isn't as important then, since
what's important to users (hardware support - kernel, and
applications) is updated independent from the development base. (As I
said, that's what many packagers - including myself - already do
within the current system.)

Fabio

Post by Brian (bex) Exelbierd
The bootable base would change based on Fedora's needs. Perhaps we
decide want to new kernels (again sorry for my failings in this field)
every 6 months to introduce new drivers and hardware support. Someone
who wants faster can self-build for their community/needs more
frequently or a hardware vendor might want a kernel that doesn't
change as often and is backported. Fedora may not build the whole
range here either.

Post by Peter Robinson

Post by Brian (bex) Exelbierd
I'd push Brendans' concept further and suggest that we try to
eliminate as many of the compilers, libraries and core system tools as
possible from this bootable-base so that those can iterate at their
own speed, perhaps 4 year for a laptop vendor and 30 day for a
experimental ARM device. Fedora as a project might not build output
for the whole range, but a build system that allowed us to help others
be successful would be a huge help here.

Again what do you even mean by eliminate the compilers? Also how do we
not change something core, such as a compiler, for 4 years while also
change it every 30 days?

"Eliminate the compilers" was meant to mean, make them modules or
non-bootable base components as much as possible. I realize that for
something like glibc that can be hard to impossible and for things
like Fortran a non-issue.
Communities have different needs, and every time we freeze or force
change we create challenges for those needs to be met. My point is
that by keeping our base to the "light up the machine level (another
hated idiom) and get containers/flatpaks/etc running" we can allow the
builder to focus on their community's needs or the user to get their
own too-fast/too-slow balance.
What I'd like to forego is the long "what is base" conversation. I'd
pull it back to what does it take to boot the machine and get
containers/flatpaks/modules running. Everything after that should be
flexible, even if we put down requirements and rules for the use cases
we want to tackle and the pieces we build and deliver.

Post by Peter Robinson

Post by Brian (bex) Exelbierd
I recognize that I, like most people, see the world through the lens
of my specific use case, but remember, "Fedora creates an innovative
platform for hardware, clouds, and containers that enables software
developers and community members to build tailored solutions for their
users." As long as we don't block a use cases arbitrarily and we
leave room for that innovation and service we are doing the right
thing. The debate about what use cases should be done fully by
Fedora, enabled for a SIG/WG via our build system or done externally
by those using only the parts that make sense for them is a separate
debate.

I agree on some of your points above, but TBH most of it reads like
some form of marketing coolaid!

Marketing Koolaid (in the interesting and love sense, not the
pejorative) would be really good for Fedora right now, if we want
increased adoption and contribution. Flexibility to allow our mission
statement to keep being true is critical. I fully admit that my
skills are not in distribution building and that you and others in
this thread have those skills in greater capacity than I do. But I
don't hear anyone starting from the premise of our mission statement
and then moving forward. I feel like a build/distribution focused on
easing the build process and ability to provide across the full
too-fast/too-slow spectrum, even if we as a project don't fill the
whole spectrum, is crucial to Fedora's ongoing success.
regards,
bex

Post by Peter Robinson
Also it does account at all for any and/or all of the resources that
we'd need to even enact some of this, even if it's possible?

Post by Brian (bex) Exelbierd
0: Builder's desires are the desires of the person who put the entire
system together to fulfill the needs of their community per our
mission statement. If my mythical llama herders need the oldest Libre
Office possible but the newest Rust packaging and whatever random
version of httpd that Fedora deems "stable", then that is what I
desire, even if the upstream or other non-llama herding users desire
something different. However, I'd also push that we should try to
reach a point where if a llama herder for non-llama reasons needs a
different httpd, they can just enable and use it (using the language
of modularity). In case it isn't clear, the "builder's desires"
includes the goals of every current lab, spin, and edition, separately
and where appropriate together.
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

--
Fedora Community Action & Impact Coordinator
@bexelbie | http://www.winglemeyer.org
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/arc

[Paul Frields]

Post by Fabio Valentini
I think the kernel is a bad example here. It's exceedly stable across
releases, so it's probably the one component that's least problematic
to upgrade during a fedora release. The fedora kernel team is already
doing that, and they are doing an excellent job, which they definitely
deserve more praise for than they get. For me, the frequent,
well-executed stable kernel updates are one thing that positively
distinguishes fedora from other non-rolling distros.

[...snip...]

I wanted to offer a hearty +1 here. I have been thinking more about
the optimization needed for a faster release process, which would
*not* disadvantage our awesome kernel team.

I wouldn't foresee, for example, that we'd freeze on a specific kernel
for such a process. On the contrary, the kernel team does an *amazing*
job making sure that Fedora is relevant to the upstream kernel
community as much as possible. We routinely get fresh kernels with
updated hardware support. I wouldn't want to see this stop.

I understand the lifecycle goals theoretically also make possible a
longer term release. How that would happen should be based on what its
consumers need, and (maybe even more importantly) what our maintainers
are able to do without each being issued a Fedora Time Turner[1]. I
don't want to optimize for that case because a much shorter cycle
(even with less fanfare) encourages us to pursue more automation and
more contributor empowerment.


* * *
[1] http://harrypotter.wikia.com/wiki/Time-Turner
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedor

[Owen Taylor]

On Wed, Nov 28, 2018 at 3:07 AM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd
I think that Fedora's role as an innovater in the OS space means we
should be aggressively exploring this. Rolling Releases, Tech-Driven
Releases and Time-Based Releases all have well known positives and
negatives. All of the work that has been done on Modularity,
containers, flatpaks, OSTree, and more, gives us the opportunity to
really re-think this. While it is true there are dozens (or more)
additional solutions to the too-fast/too-slow and the
incompatible-libraries problems, these technologies seem to be gaining
the most adoption across a variety of use cases. They are all also
generally well supported in Fedora and we need to aggressively push
them to achieve this goal or determine where the dead-end is so we can
move to the next innovation.

We have a lot of flexibility with all these technologies. What we need
to achieve through our processes - CI, the release process, etc, is to
enable users to take advantage of the technologies to achieve their
goals - to get their work done - without having new problems thrown at
them - excessive complexity, instability, etc.

Post by Brian (bex) Exelbierd
I personal am hugely in favor of us adopting a bootable-base model
that allows us to iterate the kernel and the various user-space pieces
at the speed of the upstream, the user's desires and the builder's
desires[^0] all at the same time. While this will require us to do
some level of NxM matrix building and testing, a base that didn't have
to change often for most use cases reduces the matrix considerably.

The term "bootable-base" concerns me - because it could be interpreted
to mean a minimal bootable set - just enough to get systemd and dnf
running, perhaps. In another mail, you explained what you meant as
"light up the machine level [...] and get containers/flatpaks/etc
running". What does it take to get containers running usefully? It
probably looks quite a bit like Fedora CoreOS. What does it take to
get Flatpaks running in a useful way to users? - you need a full
desktop environment - something like Fedora Silverblue (or the default
install set of Fedora Workstation if you prefer loose packages).

While the Fedora release today contains a lot more software than this
"core operating system" - generally our release processes today are
targeted at trying to release a stable core operating system - and
this is something we can't lose if we start changing around how things
work. It's not that I don't think the maintenance of glibc and gcc is
really important, but if we refocus our release processes only on
that, and one day a broken wpa-supplicant rebase lands and half our
users lose networking - that would be a really poor outcome for
Fedora.

Post by Brian (bex) Exelbierd
I'd push Brendans' concept further and suggest that we try to
eliminate as many of the compilers, libraries and core system tools as
possible from this bootable-base so that those can iterate at their
own speed, perhaps 4 year for a laptop vendor and 30 day for a
experimental ARM device. Fedora as a project might not build output
for the whole range, but a build system that allowed us to help others
be successful would be a huge help here.

Fedora needs to be an operating system provider, not just an operating
system toolbox provider. There are a lot of use cases for the
"operating system toolbox"

- To build the operating systems that Fedora releases - CoreOS,
Workstation, etc.
- To create the containers, Flatpak runtimes, and Flatpaks that Fedora releases
- For users to create application containers
- To create development environments (pet containers)
- For adventurous users to create new, customized operating systems

but you can't do integration testing on operating system toolbox,
because the whole point is that you can integrated it any way you want
and only the end user defines what is broken and what is working.
Producing a *quality* operating system requires us to focus our
processes on the integrated operating systems that we produce, on the
applications we produce - and that's what our processes should be
around.

Could we replace the current 6 month release processes with an
alternate set of processes around multiple rolling branches? It's
certainly a possibility, but it's going to be very difficult to get a
high level of stability - a level of stability that is acceptable to
all end users - without a process where the *entire operating system*
gets beta tested before being pushed out.

If we want to move in the direction of rolling releases, the first
step is to get the continuous integration testing and gating in place
to make rawhide consumable by a broader set of people. That would be
incredibly useful even if we keep the current 6 month tempo, but
essential if we want to move to a rolling stable model. Until we can
demonstrate that, it seems really premature to talk about rolling
stable models.

Owen
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archiv

[Stephen John Smoogen]

Post by Owen Taylor
On Wed, Nov 28, 2018 at 3:07 AM Brian (bex) Exelbierd

Post by Brian (bex) Exelbierd
I'd push Brendans' concept further and suggest that we try to
eliminate as many of the compilers, libraries and core system tools as
possible from this bootable-base so that those can iterate at their
own speed, perhaps 4 year for a laptop vendor and 30 day for a
experimental ARM device. Fedora as a project might not build output
for the whole range, but a build system that allowed us to help others
be successful would be a huge help here.

Fedora needs to be an operating system provider, not just an operating
system toolbox provider. <cut>

I feel like we have been saying this for 15+ years even before Fedora
was Fedora. Even back in the RHL days, we would argue over whether
what we were providing was an 'OS' or not versus a toolkit for someone
else to work with. [Especially during the days when Mandrake and a
couple others were just rebuilds of Red Hat Linux with specific kernel
flags and some light patching to sed Red Hat Linux to Mandrake.]

Do we even know what an operating system provider is? Because it seems
like we do this every release where each groups look at what was
produced versus what they wanted and all they can see is all the
compromises they had to make to get it out the door. And unless we
actually come up with some specific things of what is really wanted,
we will be doing this every release in the future too.


--
Stephen J Smoogen.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedorapro

[Owen Taylor]

Post by Stephen John Smoogen

Post by Owen Taylor
Fedora needs to be an operating system provider, not just an operating
system toolbox provider. <cut>

I feel like we have been saying this for 15+ years even before Fedora
was Fedora. Even back in the RHL days, we would argue over whether
what we were providing was an 'OS' or not versus a toolkit for someone
else to work with.

I don't think we've just been saying this, I think we've been steadily
improving in this area - both in our focus and in our processes. The
move to editions was particularly helpful.

Owen
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@list

[Stephen John Smoogen]

Post by Owen Taylor

Post by Stephen John Smoogen

Post by Owen Taylor
Fedora needs to be an operating system provider, not just an operating
system toolbox provider. <cut>

I feel like we have been saying this for 15+ years even before Fedora
was Fedora. Even back in the RHL days, we would argue over whether
what we were providing was an 'OS' or not versus a toolkit for someone
else to work with.

I don't think we've just been saying this, I think we've been steadily
improving in this area - both in our focus and in our processes. The
move to editions was particularly helpful.

I didn't mean to assume we haven't improved on it, but I do believe it
is the central 'conflict' we have had. My main worry is that can we
actually achieve being an "operating system provider" or are we always
going to be a toolbox who isn't happy with itself? (or thirdly, are we
already an operating system provider with delusions of being a
toolbox?)

How can we know if we actually are still making progress if we each
have a different definition of what that operating system is.

Post by Owen Taylor
Owen
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

--
Stephen J Smoogen.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/ar

[Josh Boyer]

Post by Stephen John Smoogen

Post by Owen Taylor

Post by Stephen John Smoogen

Post by Owen Taylor
Fedora needs to be an operating system provider, not just an operating
system toolbox provider. <cut>

I feel like we have been saying this for 15+ years even before Fedora
was Fedora. Even back in the RHL days, we would argue over whether
what we were providing was an 'OS' or not versus a toolkit for someone
else to work with.

I don't think we've just been saying this, I think we've been steadily
improving in this area - both in our focus and in our processes. The
move to editions was particularly helpful.

I didn't mean to assume we haven't improved on it, but I do believe it
is the central 'conflict' we have had. My main worry is that can we
actually achieve being an "operating system provider" or are we always
going to be a toolbox who isn't happy with itself? (or thirdly, are we
already an operating system provider with delusions of being a
toolbox?)

Why do either of those possibilities matter? Or asked more directly,
why wouldn't we continue to look at ourselves and iterate and try to
improve? There is no "done" state in an operating system.

Post by Stephen John Smoogen
How can we know if we actually are still making progress if we each
have a different definition of what that operating system is.

Great question. I know there are different ideas around it in terms
of platform vs. core/extras vs. lifecycles of different bits. I think
coming to a general consensus on that is a huge part of the discussion
we're having here.

josh
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorapro

[Stephen John Smoogen]

Post by Josh Boyer

Post by Stephen John Smoogen

Post by Owen Taylor

Post by Stephen John Smoogen

Post by Owen Taylor
Fedora needs to be an operating system provider, not just an operating
system toolbox provider. <cut>

I feel like we have been saying this for 15+ years even before Fedora
was Fedora. Even back in the RHL days, we would argue over whether
what we were providing was an 'OS' or not versus a toolkit for someone
else to work with.

I don't think we've just been saying this, I think we've been steadily
improving in this area - both in our focus and in our processes. The
move to editions was particularly helpful.

I didn't mean to assume we haven't improved on it, but I do believe it
is the central 'conflict' we have had. My main worry is that can we
actually achieve being an "operating system provider" or are we always
going to be a toolbox who isn't happy with itself? (or thirdly, are we
already an operating system provider with delusions of being a
toolbox?)

Why do either of those possibilities matter? Or asked more directly,
why wouldn't we continue to look at ourselves and iterate and try to
improve? There is no "done" state in an operating system.

You communicated clearer in your previous email what I wanted to say
better than I have in mine. Evolution versus revolution. We spend a
lot of time and energy talking for and against revolutions every
release because we don't seem to know what we are at and when in fact
most of them are evolutions which later on weren't as big as expected.

I just want us to come to some consensus on what we have accomplished,
where we are at, and then see if we can work out what we need to
iterate and improve. [I am probably still not communicating this any
better than I did in my previous emails, as I think you covered it
very well previously. ]

Post by Josh Boyer

Post by Stephen John Smoogen
How can we know if we actually are still making progress if we each
have a different definition of what that operating system is.

Great question. I know there are different ideas around it in terms
of platform vs. core/extras vs. lifecycles of different bits. I think
coming to a general consensus on that is a huge part of the discussion
we're having here.

Thanks. I am interested in working out what we have accomplished so
far, and what we are hurting on in concrete terms.

Post by Josh Boyer
josh
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

--
Stephen J Smoogen.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedorapr

[Josh Boyer]

Post by Owen Taylor

Post by Stephen John Smoogen

Post by Owen Taylor
Fedora needs to be an operating system provider, not just an operating
system toolbox provider. <cut>

I feel like we have been saying this for 15+ years even before Fedora
was Fedora. Even back in the RHL days, we would argue over whether
what we were providing was an 'OS' or not versus a toolkit for someone
else to work with.

I don't think we've just been saying this, I think we've been steadily
improving in this area - both in our focus and in our processes. The
move to editions was particularly helpful.

I think this is a key observation. Some people thought Editions were
just rearranging stuff arbitrarily or for little value. I continue to
see them as a great way to focus on specific user bases. What we're
discussing now is just continuation of that at a more fundamental
level.

For some reason I can't understand, people want a revolution every
time. Fedora often has a tendency to look at evolution as failure.
That isn't sustainable and leads to repeating mistakes, creating
totally new problems, etc. There is nothing wrong with iteration as
long as you know where you're going and why you're making the changes
you're making.

In this particular round, I think focusing on some of the fundamentals
on how we construct our OS (compose tooling, real actual CI, etc) is
another evolution. Does it materially change what Fedora is to end
users? No. Does it have benefits for them and for maintainers in the
long run? Yes.

We seem to also keep trying to force fit a single OS release
methodology into all use cases. That leads to a poor fit many times.
With looking at multiple lifecycles and how to pull that off using the
fundamentals above, we can actually still look at being an OS provider
but it doesn't have to be a singular OS. We can create the platform
necessary to have commonality at certain layers across different use
cases.

Do I think all of this will happen in a single Fedora "release"? No.
It's good to be somewhat timeboxed, and I think we can make really
great strides on some of it. For other bits we'll have to see how
things fall out going forward. But if we don't take the time now to
work on the enablers, the big picture or theoretical goals won't
matter.

josh
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/l

[Paul Frields]

[...snip...]

Post by Josh Boyer

Post by Owen Taylor

Post by Josh Boyer
I completely disagree. Our release process and tooling is built on
heroism and tech debt. At some point, that is going to cause
significant burnout and then people will just wonder what happened
when those people stop doing releases. I think it's better to raise
awareness at the project level, and build something that is
sustainable rather than predicated on "small group of people killing
themselves for the greater good". That starts with individual package
CI gating, and goes all the way through integration testing between
packages in a gated manner, into how we actually *create* all of that
plus the things we deem worth of releasing.

I think you are misunderstanding me somehow. I'm 100% on-board with
improving the processes, catching compose problems in an automated and
early fashion, making developers fix their own problems, and generally
making releases a less heroic process. And if that requires a
temporary cadence chance to get the retooling done, then it's worth
it.

Ah, good!

Post by Owen Taylor
But Brendan's proposal to permanently slow down the cadence seems to
make the assumption that the current release cycle is a heroic effort
for the entire project - that we're moving too fast to stay on our
feet. And I don't think that's the case.

That would be a concern to me as well, assuming we stuck with a
*single* cadence and a *single* platform. With the lifecycle
Objective, aren't we targeting multiple cadences and lifecycles? I
mean, we have this today with Fedora Atomic/CoreOS vs. "regular"
Fedora. Maybe Brendan was thinking singular, but I would very much
like the ability in our tooling a process to have both the faster
moving cadence of today AND a slower moving one. That's another
reason I view a temporary halt as worthwhile. If done correctly, it
enables Fedora to fill more usecases and lifecycles than we could ever
accomplish with a single release.

Josh and Owen both have it basically right IMHO. Speeding up the
compose, increasing automation, and making it easier for people beyond
the core crew to do releases all should allow us to make releases less
of a big deal. I would love to get closer to the ostree release (both
in terms of content, and cycle) for the majority of current users.
That means faster, not slower releases -- and that "release" should be
more like a non-event for most people. Not to mention the benefits of
being able to revert the platform in case of a problem, etc.

But I also agree that a longer cycle can have a place too. The tricky
part is to make sure that doesn't turn into multi-stream pain for the
proverbial "1000 packagers" group. There should be a way to set the
cycle and the overlap to minimize pain. (And I suspect modularity also
helps somewhat here.)

Take an average packager who cares about long-term. They may have to
maintain 5+ streams now if you count EPEL. We want an outcome that is
arguably better, or at least no worse. (No, we really want better.)
;-)

--
Paul
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list

[Kevin Fenzi]

I agree with the folks in this subthread, but I think we are going to
have to look at 'redesigning' things more than just 'optimizing'.

ie, collect all our inputs and outputs and things we need to do in the
process and figure out how to make it modular (no relation) so we can
look at just a single changed input and quickly test and generate all
the outputs that are affected by that input (and not any of the others).

For example, a new xfce4-settings package comes out, we want to test it,
test the Xfce live media and if it all passes, perhaps release that media.

It may well be that we can figure out a 'base platform' just by looking
at these imput/outputs: what things are in most or all outputs?

kevin

[Sudhir]

Post by Kevin Fenzi
I agree with the folks in this subthread, but I think we are going to
have to look at 'redesigning' things more than just 'optimizing'.

+1. At the same time, we should also ensure that Devel,QE,Infra,RelEng..
all equal stake holders and are hand in glove with the new design and
not fall into their own silo.

Post by Kevin Fenzi
ie, collect all our inputs and outputs and things we need to do in the
process and figure out how to make it modular (no relation) so we can
look at just a single changed input and quickly test and generate all
the outputs that are affected by that input (and not any of the others).
For example, a new xfce4-settings package comes out, we want to test it,
test the Xfce live media and if it all passes, perhaps release that media.
It may well be that we can figure out a 'base platform' just by looking
at these imput/outputs: what things are in most or all outputs?
kevin
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

[Mohan Boddu]

I totally agree with Paul and Kevin.

I want to see a faster release cycle (probably rolling release) and shorter
processes to get a release out.

Post by Kevin Fenzi
I agree with the folks in this subthread, but I think we are going to
have to look at 'redesigning' things more than just 'optimizing'.
ie, collect all our inputs and outputs and things we need to do in the
process and figure out how to make it modular (no relation) so we can
look at just a single changed input and quickly test and generate all
the outputs that are affected by that input (and not any of the others).
For example, a new xfce4-settings package comes out, we want to test it,
test the Xfce live media and if it all passes, perhaps release that media.

If we can do this, we can release different artifacts at different times
which
gives more comfort and flexibility to different WG's.

Post by Kevin Fenzi
It may well be that we can figure out a 'base platform' just by looking
at these imput/outputs: what things are in most or all outputs?
kevin
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

[Miroslav Suchý]

Post by Josh Boyer

Post by Owen Taylor
In other words, the "technical debt" we are trying to solve here is
not project wide and doesn't justify slowing down the whole project
permanently.

I completely disagree. Our release process and tooling is built on
heroism and tech debt.

People working on release and people working on packages maintenance are different group - they are not disjunct, but it
is not the same group.
For example *I* am a maintainer of lots of packages, but the additional works because of the fedora release is about one
working day per year - and it is mostly because of fedora-upgrade package. Other packages do not need so much work. I am
more affected by upstream releases.

Do not forget that annual releases will mean that N-1 release will implicate 24 months support for packages which will
mean a much more significant impact on us-the maintaners.

Miroslav
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedora

[Paul Frields]

Post by Miroslav Suchý

Post by Josh Boyer

Post by Owen Taylor
In other words, the "technical debt" we are trying to solve here is
not project wide and doesn't justify slowing down the whole project
permanently.

I completely disagree. Our release process and tooling is built on
heroism and tech debt.

People working on release and people working on packages maintenance are different group - they are not disjunct, but it
is not the same group.
For example *I* am a maintainer of lots of packages, but the additional works because of the fedora release is about one
working day per year - and it is mostly because of fedora-upgrade package. Other packages do not need so much work. I am
more affected by upstream releases.
Do not forget that annual releases will mean that N-1 release will implicate 24 months support for packages which will
mean a much more significant impact on us-the maintaners.

Let's not get too focused on an annual release (or any specific
timeframe for longer release). I know this is something that some
people want. I understand that, and it's *possible* to do in a future
state where more people are empowered to make releases happen. But a
longer release is not the primary goal, merely something that's
possible.

I'm more interested in a shorter release that implies less added
maintainer effort. We already put time into Rawhide, and I would like
to see that better leveraged in what we push to consumers. Right now
our branch cycle is six months, at which point we have numerous
freezes and other operations that consume a lot of manual time. They
also bottleneck our pace.

I want to increase automation, decrease manual bottlenecks and
freezes, and spread out permissions to assemble and push out "ready"
content. I would like to optimize for a faster release, making any
slower releases possible. Those releases should be based on what the
consumers of that release need, as well as the efforts our maintainers
have available.

--
Paul
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedoraproj

[Josh Boyer]

Post by Paul Frields

Post by Miroslav Suchý

Post by Josh Boyer

Post by Owen Taylor
In other words, the "technical debt" we are trying to solve here is
not project wide and doesn't justify slowing down the whole project
permanently.

I completely disagree. Our release process and tooling is built on
heroism and tech debt.

People working on release and people working on packages maintenance are different group - they are not disjunct, but it
is not the same group.
For example *I* am a maintainer of lots of packages, but the additional works because of the fedora release is about one
working day per year - and it is mostly because of fedora-upgrade package. Other packages do not need so much work. I am
more affected by upstream releases.
Do not forget that annual releases will mean that N-1 release will implicate 24 months support for packages which will
mean a much more significant impact on us-the maintaners.

I'll echo what Paul says below with a +1, but I wanted to touch on
this point a bit because it implies an assumption that the maintenance
model remains the same even if lifecycle options change. I don't
think that needs to be the case, nor do I think it would even be good.

Of the large number of packages that you maintain, how many of them
are critical to freeze at a specific version for a given Fedora
release? Possibly some, but I would think across the distribution it
would not be a huge number. So if there is no essential need to
freeze them at a specific version, why would you want to maintain the
packages *separately* for each release? That sounds like extra work
for no benefit. If we instead take a maintenance approach that you
maintain package foo and it is built for all releases, then you only
really need to maintain it in a singular instance.

Today that is something that can be accomplished with modularity, but
I would suggest that we look at stream branching as a solution even
for regular packages. So you wouldn't have fc22-fc32 branches under
package foo. You'd have foo/stream-<version> and you could build that
wherever you'd like. Couple that with automated CI testing and I
believe you actually decrease your maintenance burden while increasing
your quality.

There are many details that would need to be worked out and I don't
want to trivialize them, but I do want to at least get people thinking
about it in the long term. If we are going to improve the way we
build and deliver our operating system, we shouldn't assume we can do
that without changing the way we maintain it either.

josh

Post by Paul Frields
Let's not get too focused on an annual release (or any specific
timeframe for longer release). I know this is something that some
people want. I understand that, and it's *possible* to do in a future
state where more people are empowered to make releases happen. But a
longer release is not the primary goal, merely something that's
possible.
I'm more interested in a shorter release that implies less added
maintainer effort. We already put time into Rawhide, and I would like
to see that better leveraged in what we push to consumers. Right now
our branch cycle is six months, at which point we have numerous
freezes and other operations that consume a lot of manual time. They
also bottleneck our pace.
I want to increase automation, decrease manual bottlenecks and
freezes, and spread out permissions to assemble and push out "ready"
content. I would like to optimize for a faster release, making any
slower releases possible. Those releases should be based on what the
consumers of that release need, as well as the efforts our maintainers
have available.
--
Paul
_______________________________________________
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines

_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@l

[Neal Gompa]

Post by Josh Boyer

Post by Miroslav Suchý

Post by Josh Boyer

Post by Owen Taylor
In other words, the "technical debt" we are trying to solve here is
not project wide and doesn't justify slowing down the whole project
permanently.

I completely disagree. Our release process and tooling is built on
heroism and tech debt.

People working on release and people working on packages maintenance are different group - they are not disjunct, but it
is not the same group.
For example *I* am a maintainer of lots of packages, but the additional works because of the fedora release is about one
working day per year - and it is mostly because of fedora-upgrade package. Other packages do not need so much work. I am
more affected by upstream releases.
Do not forget that annual releases will mean that N-1 release will implicate 24 months support for packages which will
mean a much more significant impact on us-the maintaners.

I'll echo what Paul says below with a +1, but I wanted to touch on
this point a bit because it implies an assumption that the maintenance
model remains the same even if lifecycle options change. I don't
think that needs to be the case, nor do I think it would even be good.
Of the large number of packages that you maintain, how many of them
are critical to freeze at a specific version for a given Fedora
release? Possibly some, but I would think across the distribution it
would not be a huge number. So if there is no essential need to
freeze them at a specific version, why would you want to maintain the
packages *separately* for each release? That sounds like extra work
for no benefit. If we instead take a maintenance approach that you
maintain package foo and it is built for all releases, then you only
really need to maintain it in a singular instance.
Today that is something that can be accomplished with modularity, but
I would suggest that we look at stream branching as a solution even
for regular packages. So you wouldn't have fc22-fc32 branches under
package foo. You'd have foo/stream-<version> and you could build that
wherever you'd like. Couple that with automated CI testing and I
believe you actually decrease your maintenance burden while increasing
your quality.
There are many details that would need to be worked out and I don't
want to trivialize them, but I do want to at least get people thinking
about it in the long term. If we are going to improve the way we
build and deliver our operating system, we shouldn't assume we can do
that without changing the way we maintain it either.

We can change this _today_, actually. fedpkg supports an in-repo
config file to specify distro targets to push whenever running `fedpkg
build`. So you could do a repo with only a master branch and have it
push to all distro targets enabled for the repo at once. This is
probably a useful optimization for the overwhelming majority of
packages held by packagers.

It's just not documented or available as an option for setup when you
file a repo creation request.


--
真実はいつも一つ！/ Always, there's only one truth!
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archive

[Josh Boyer]

Post by Neal Gompa

Post by Josh Boyer

Post by Miroslav Suchý

Post by Josh Boyer

Post by Owen Taylor
In other words, the "technical debt" we are trying to solve here is
not project wide and doesn't justify slowing down the whole project
permanently.

I completely disagree. Our release process and tooling is built on
heroism and tech debt.

People working on release and people working on packages maintenance are different group - they are not disjunct, but it
is not the same group.
For example *I* am a maintainer of lots of packages, but the additional works because of the fedora release is about one
working day per year - and it is mostly because of fedora-upgrade package. Other packages do not need so much work. I am
more affected by upstream releases.
Do not forget that annual releases will mean that N-1 release will implicate 24 months support for packages which will
mean a much more significant impact on us-the maintaners.

I'll echo what Paul says below with a +1, but I wanted to touch on
this point a bit because it implies an assumption that the maintenance
model remains the same even if lifecycle options change. I don't
think that needs to be the case, nor do I think it would even be good.
Of the large number of packages that you maintain, how many of them
are critical to freeze at a specific version for a given Fedora
release? Possibly some, but I would think across the distribution it
would not be a huge number. So if there is no essential need to
freeze them at a specific version, why would you want to maintain the
packages *separately* for each release? That sounds like extra work
for no benefit. If we instead take a maintenance approach that you
maintain package foo and it is built for all releases, then you only
really need to maintain it in a singular instance.
Today that is something that can be accomplished with modularity, but
I would suggest that we look at stream branching as a solution even
for regular packages. So you wouldn't have fc22-fc32 branches under
package foo. You'd have foo/stream-<version> and you could build that
wherever you'd like. Couple that with automated CI testing and I
believe you actually decrease your maintenance burden while increasing
your quality.
There are many details that would need to be worked out and I don't
want to trivialize them, but I do want to at least get people thinking
about it in the long term. If we are going to improve the way we
build and deliver our operating system, we shouldn't assume we can do
that without changing the way we maintain it either.

We can change this _today_, actually. fedpkg supports an in-repo
config file to specify distro targets to push whenever running `fedpkg
build`. So you could do a repo with only a master branch and have it
push to all distro targets enabled for the repo at once. This is
probably a useful optimization for the overwhelming majority of
packages held by packagers.

Yep, I know :) For a lot of packages, this could be done now. I
think to get the full benefits from it across the distro, we'd need to
really define that platform layer so maintainers know what they can
depend on, etc.

Post by Neal Gompa
It's just not documented or available as an option for setup when you
file a repo creation request.

Right. I think we should start encouraging its use. I kind of
dislike using 'master' as the branch because it's ambiguous depending
on how you look at it, but it's not wrong. A stream branch just
provides a bit more context.

josh
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedoraproject.o

[Nicolas Mailhot]

Post by Josh Boyer
Of the large number of packages that you maintain, how many of them
are critical to freeze at a specific version for a given Fedora
release? Possibly some, but I would think across the distribution it
would not be a huge number.

It's not so much that many packages need freezing, but quite a lot need
coordinated rebuilds (mini mass rebuilds). Once thing current cadencing
provides is "free" mass rebuilds (you just make sure to commit the key
packages before branching and releng will usually mass rebuild all the
other things that depend on those key parts for some other reason).

Fix the tooling so those mini mass rebuilds are cheap to setup and are
not human packager intensive, and there's not reason the rebuild result
could not be pushed to every release. Or to a rolling release. Or
whatever.

We really need to evolve our tooling from "packages can be handled
independently in reviews, builds, and pushes, coordinated changes are
the exception" to "we manage sets of packages by default, single-package
changes are just set changes with set = 1"

--
Nicolas Mailhot
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/arc

[Josh Boyer]

On Thu, Nov 29, 2018 at 12:42 PM Nicolas Mailhot

Post by Nicolas Mailhot

Post by Josh Boyer
Of the large number of packages that you maintain, how many of them
are critical to freeze at a specific version for a given Fedora
release? Possibly some, but I would think across the distribution it
would not be a huge number.

It's not so much that many packages need freezing, but quite a lot need
coordinated rebuilds (mini mass rebuilds). Once thing current cadencing
provides is "free" mass rebuilds (you just make sure to commit the key
packages before branching and releng will usually mass rebuild all the
other things that depend on those key parts for some other reason).
Fix the tooling so those mini mass rebuilds are cheap to setup and are
not human packager intensive, and there's not reason the rebuild result
could not be pushed to every release. Or to a rolling release. Or
whatever.

Yes, agreed. Fixing the tooling is what much of these threads are about ;)

Post by Nicolas Mailhot
We really need to evolve our tooling from "packages can be handled
independently in reviews, builds, and pushes, coordinated changes are
the exception" to "we manage sets of packages by default, single-package
changes are just set changes with set = 1"

Modularity does that for a number of cases, but it isn't a silver
bullet. Our package dependencies web is much too complex to make it
feasible for everything to be in a module, at least right now.

(We should really also revisit our package dependencies in general and
try to minimize as much as we can for a variety of reasons, leveraging
rich/weak deps. That's a different topic though.)

josh
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedoraproject