Maintenance: use AWK to generate release notes

[yadij]

While sed can perform what we need currently AWK will allow some
more advanced operations like nested file include with variable
substitutions.

[yadij]

This is a drop-in replacement. No output changes from current official code.

[rousskov]

AWK will allow some more advanced operations like nested file include with variable
substitutions.

Be as it may, there are overall better languages/tools than AWK to handle most "advanced operations", and there appears to be no consensus regarding the future of release notes generation (i.e. we do not know what those "advanced operations" are going to be), with at least one of the proposals on the table definitely requiring higher level language/tools than AWK.

We should develop that long-term design consensus before deciding what tools to use to implement that design. A recent #2261 (comment) thread attempts to do that (and is waiting for your response).

[kinkie]

Isn't this the same as
$(FGREP) "@SQUID" $@ >/dev/null
?

[yadij]

I'm not sure we need that line at all really. Especially now that the || ... handler exists. If we agree to remove or change it I am fine with that. Otherwise I will leave out of scope.

[kinkie]

TBH this code looks a bit overengineered and at best harder to read than the code it replaces.
What kind of need do you foresee for the more advanced operations mentioned in the PR comment?

[yadij]

TBH this code looks a bit overengineered and at best harder to read than the code it replaces. What kind of need do you foresee for the more advanced operations mentioned in the PR comment?

I was looking into @rousskov's request to put notes about each feature into separate files. Seeing how far we could go before the obvious problems arise.

The next steps I have been looking at are:
3) using the version number to determine whether or not the Notice section text that is supposed to be exclusively beta release text gets added.
4) using the template.sgml.in as input to assemble from instead of release-N.sgml.in
5) moving the Known Issues section into separate file. Each vN branch would have its own content there.
6) adding a file documenting the Build Requirements section which has come and gone over releases. Each vN branch would have its own content there.

At this point I still foresee all the "impossible to solve" problems occurring with "Major New Features", squid.conf, and ./configure sections being edited on a per-branch basis. However, the framework is there for Alex to show us what model he is thinking somehow solves it magically - and we can discuss specific code instead of debating ideals for more years.

[yadij]

We should develop that long-term design consensus before deciding what tools to use to implement that design. A recent #2261 (comment) thread attempts to do that (and is waiting for your response).

My input to the discussion is the same. I do not believe your plan will work at all. Here is a framework you can use to prove me wrong though.

FYI: I picked AWK since the logic syntax is the most C-ish, POSIX shell is also guaranteed to be available, Perl is a maybe. Whatever script is used here it must run on content from the distributed tarball (ie no git) because downstream apply their own patches then re-make dist inside secure jail environments.

[rousskov]

We should develop that long-term design consensus before deciding what tools to use to implement that design. A recent #2261 (comment) thread attempts to do that (and is waiting for your response).

My input to the discussion is the same. I do not believe your plan will work at all.

That belief is not the blocking question right now. The current blocking question is whether you are going to veto an implementation of what I have proposed if such implementation is posted as a PR¹. If you intend to veto it (based on your believes or any other reason derived from the proposal itself), then there is no point in implementing my proposal!

Here is a framework you can use to prove me wrong though.

If "Here" is this PR, then this PR is not a framework, and it cannot be used to prove you wrong (or prove you right) with regard to my proposal viability.

Whatever script is used here it must run on content from the distributed tarball (ie no git)

I strongly disagree.

because downstream apply their own patches then re-make dist inside secure jail environments.

We do not need to accommodate "downstream" that cannot use git, but even such "downstream" are free to patch artifacts that we produce in a git-enabled environment.

Footnotes

1. Obviously, if some deadly flaws are discovered during my proposal implementation, then its implementation will fail and the proposal will be withdrawn. ↩

[rousskov]

I was looking into @rousskov's request to put notes about each feature into separate files.

FWIW, I see no relationship between this PR and my proposal.

The next steps I have been looking at are:

3. using the version number to determine whether or not the Notice section text that is supposed to be exclusively beta release text gets added.

Most likely, the Notice section should be removed. If we fail to agree on that, it should probably become a separate versioned file (that gets assembled into a release notes document) and it should be adjusted to tailor release notes needs instead of "announcement" needs.

4. using the template.sgml.in as input to assemble from instead of release-N.sgml.in

I disagree that we should use a hard-coded template to assemble a release notes document. In a sense, the assembling program should become that template instead because various indexing/placement decisions depend on a dynamic set of notes. If some static template survives, it will be very minimal/basic.

5. moving the Known Issues section into separate file. Each vN branch would have its own content there.

We should not duplicate bug reports so, most likely, the Known Issues section should be removed or replaced with a simple reference to Bugzilla/etc. If we fail to agree on that, in my proposal, each "known issue" (if any) would be a separate versioned file, just like each "change note".

6. adding a file documenting the Build Requirements section which has come and gone over releases. Each vN branch would have its own content there.

A list of build requirements seems to be unrelated to release notes. Each change in build requirements would become a versioned file in my proposal, assembled into a release notes document.

At this point I still foresee all the "impossible to solve" problems occurring with "Major New Features", squid.conf, and ./configure sections being edited on a per-branch basis. However, the framework is there for Alex to show us what model he is thinking somehow solves it magically - and we can discuss specific code instead of debating ideals for more years.

I do not see a framework. As for "specific code" in this PR, I have reviewed and blocked it; I do not recommend working on that specific code further because my block reasons are "fundamental" rather than specific to lower-level code bugs.

It is up to us, but I, personally, do not think we should be debating ideals for years, and I see no signs that we disagree on ideals in this specific context. My proposal already "shows us what model" solves many of the correct problems (rather than all problems, of course). I do not know what else you want me to "show".

[yadij]

I was looking into @rousskov's request to put notes about each feature into separate files.

FWIW, I see no relationship between this PR and my proposal.

The next steps I have been looking at are:

3. using the version number to determine whether or not the Notice section text that is supposed to be exclusively beta release text gets added.

Most likely, the Notice section should be removed. If we fail to agree on that, it should probably become a separate versioned file (that gets assembled into a release notes document) and it should be adjusted to tailor release notes needs instead of "announcement" needs.

4. using the template.sgml.in as input to assemble from instead of release-N.sgml.in

I disagree that we should use a hard-coded template to assemble a release notes document. In a sense, the assembling program should become that template instead because various indexing/placement decisions depend on a dynamic set of notes. If some static template survives, it will be very minimal/basic.

So instead of taking a single refactor step, you would just jump to a clean-slate implementation of the entire release notes creation process, design, and implementation logic.
... Sure, should only take you a few minutes, right? no new bugs, right? Hmm.

5. moving the Known Issues section into separate file. Each vN branch would have its own content there.

We should not duplicate bug reports so, most likely, the Known Issues section should be removed or replaced with a simple reference to Bugzilla/etc. If we fail to agree on that, in my proposal, each "known issue" (if any) would be a separate versioned file, just like each "change note".

Um. Please take a look at what past release notes contain for "Known Issues". I make no proposal about listing bugs or changing what it has - simply acknowledge that vN branches may have different content.

6. adding a file documenting the Build Requirements section which has come and gone over releases. Each vN branch would have its own content there.

A list of build requirements seems to be unrelated to release notes. Each change in build requirements would become a versioned file in my proposal, assembled into a release notes document.

One file that differs between vN branches would be entirely sufficient. I imagine we will settle on the same style as "Known Issues" currently uses. eg. a reference to the wiki page with where full details can be found, plus a few specific mentions when noteworthy changes happen since the vN-1.

At this point I still foresee all the "impossible to solve" problems occurring with "Major New Features", squid.conf, and ./configure sections being edited on a per-branch basis. However, the framework is there for Alex to show us what model he is thinking somehow solves it magically - and we can discuss specific code instead of debating ideals for more years.

I do not see a framework.

Maybe a difference in our understanding of the paradigm behind the term. AIUI, a place where you can implement and "drop in" arbitrary business logic (in the form of an AWK script) fits the term "framework" nicely.

As for "specific code" in this PR, I have reviewed and blocked it; I do not recommend working on that specific code further because my block reasons are "fundamental" rather than specific to lower-level code bugs.

It is up to us, but I, personally, do not think we should be debating ideals for years, and I see no signs that we disagree on ideals in this specific context. My proposal already "shows us what model" solves many of the correct problems (rather than all problems, of course). I do not know what else you want me to "show".

We have a significant disagreement on what "problems" exist in relation to the release notes.