Merge bitcoin/bitcoin#24263: doc: Fix gen-manpages, rewrite in Python

87f54060ff doc: Swap gen-manpages and update RC steps in release process (laanwj)
42c202893b doc: Fix gen-manpages, rewrite in Python (laanwj)

Pull request description:

  Rewrite the manual page generation script in Python.

  This:
  - solves '-' stripping issue (fixes #22681)
  - makes that a copyright footer is generated correctly again

  Also change the release process to swap gen-manpages and update RC steps, so that the pages will have the correct rc and/or final version.

ACKs for top commit:
  dongcarl:
    Code Review ACK 87f54060ff
  fanquake:
    ACK 87f54060ff - tested generating and opening the man pages locally, but didn't run through the release process. Will propose some changes to address consolidating the help / version output.

Tree-SHA512: 39254721ca84e4f223a321c554f2e08c36428b15019a0f9fa3eff408b4c6f1e1d74941143f4d2927427afa3ad7a7e6f999d6ec660132d817809b640a87ae9f7d
pull/24406/head
fanquake 3 years ago
commit bd6b1d0238
No known key found for this signature in database
GPG Key ID: 2EEB9F5CC09526C1

@ -76,7 +76,7 @@ year rather than two hyphenated years.
If the file already has a copyright for `The Bitcoin Core developers`, the
script will exit.
gen-manpages.sh
gen-manpages.py
===============
A small script to automatically create manpages in ../../doc/man by running the release binaries with the -help option.
@ -87,7 +87,7 @@ repostitory. To use this tool with out-of-tree builds set `BUILDDIR`. For
example:
```bash
BUILDDIR=$PWD/build contrib/devtools/gen-manpages.sh
BUILDDIR=$PWD/build contrib/devtools/gen-manpages.py
```
security-check.py and test-security-check.py

@ -0,0 +1,73 @@
#!/usr/bin/env python3
# Copyright (c) 2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
import os
import subprocess
import sys
import tempfile
BINARIES = [
'src/bitcoind',
'src/bitcoin-cli',
'src/bitcoin-tx',
'src/bitcoin-wallet',
'src/bitcoin-util',
'src/qt/bitcoin-qt',
]
# Paths to external utilities.
git = os.getenv('GIT', 'git')
help2man = os.getenv('HELP2MAN', 'help2man')
# If not otherwise specified, get top directory from git.
topdir = os.getenv('TOPDIR')
if not topdir:
r = subprocess.run([git, 'rev-parse', '--show-toplevel'], stdout=subprocess.PIPE, check=True, universal_newlines=True)
topdir = r.stdout.rstrip()
# Get input and output directories.
builddir = os.getenv('BUILDDIR', topdir)
mandir = os.getenv('MANDIR', os.path.join(topdir, 'doc/man'))
# Verify that all the required binaries are usable, and extract copyright
# message in a first pass.
copyright = None
versions = []
for relpath in BINARIES:
abspath = os.path.join(builddir, relpath)
try:
r = subprocess.run([abspath, '--version'], stdout=subprocess.PIPE, universal_newlines=True)
except IOError:
print(f'{abspath} not found or not an executable', file=sys.stderr)
sys.exit(1)
# take first line (which must contain version)
verstr = r.stdout.split('\n')[0]
# last word of line is the actual version e.g. v22.99.0-5c6b3d5b3508
verstr = verstr.split()[-1]
assert verstr.startswith('v')
# Only bitcoin-qt prints the copyright message on --version, so store it specifically.
if relpath == 'src/qt/bitcoin-qt':
copyright = r.stdout.split('\n')[1:]
versions.append((abspath, verstr))
if any(verstr.endswith('-dirty') for (_, verstr) in versions):
print("WARNING: Binaries were built from a dirty tree.")
print('man pages generated from dirty binaries should NOT be committed.')
print('To properly generate man pages, please commit your changes (or discard them), rebuild, then run this script again.')
print()
with tempfile.NamedTemporaryFile('w', suffix='.h2m') as footer:
# Create copyright footer, and write it to a temporary include file.
assert copyright
footer.write('[COPYRIGHT]\n')
footer.write('\n'.join(copyright).strip())
footer.flush()
# Call the binaries through help2man to produce a manual page for each of them.
for (abspath, verstr) in versions:
outname = os.path.join(mandir, os.path.basename(abspath) + '.1')
print(f'Generating {outname}')
subprocess.run([help2man, '-N', '--version-string=' + verstr, '--include=' + footer.name, '-o', outname, abspath], check=True)

@ -1,53 +0,0 @@
#!/usr/bin/env bash
# Copyright (c) 2016-2021 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C
TOPDIR=${TOPDIR:-$(git rev-parse --show-toplevel)}
BUILDDIR=${BUILDDIR:-$TOPDIR}
BINDIR=${BINDIR:-$BUILDDIR/src}
MANDIR=${MANDIR:-$TOPDIR/doc/man}
BITCOIND=${BITCOIND:-$BINDIR/bitcoind}
BITCOINCLI=${BITCOINCLI:-$BINDIR/bitcoin-cli}
BITCOINTX=${BITCOINTX:-$BINDIR/bitcoin-tx}
WALLET_TOOL=${WALLET_TOOL:-$BINDIR/bitcoin-wallet}
BITCOINUTIL=${BITCOINQT:-$BINDIR/bitcoin-util}
BITCOINQT=${BITCOINQT:-$BINDIR/qt/bitcoin-qt}
[ ! -x "$BITCOIND" ] && echo "$BITCOIND not found or not executable." && exit 1
# Don't allow man pages to be generated for binaries built from a dirty tree
DIRTY=""
for cmd in $BITCOIND $BITCOINCLI $BITCOINTX $WALLET_TOOL $BITCOINUTIL $BITCOINQT; do
VERSION_OUTPUT=$($cmd --version)
if [[ $VERSION_OUTPUT == *"dirty"* ]]; then
DIRTY="${DIRTY}${cmd}\n"
fi
done
if [ -n "$DIRTY" ]
then
echo -e "WARNING: the following binaries were built from a dirty tree:\n"
echo -e "$DIRTY"
echo "man pages generated from dirty binaries should NOT be committed."
echo "To properly generate man pages, please commit your changes to the above binaries, rebuild them, then run this script again."
fi
# The autodetected version git tag can screw up manpage output a little bit
read -r -a BTCVER <<< "$($BITCOINCLI --version | head -n1 | awk -F'[ -]' '{ print $6, $7 }')"
# Create a footer file with copyright content.
# This gets autodetected fine for bitcoind if --version-string is not set,
# but has different outcomes for bitcoin-qt and bitcoin-cli.
echo "[COPYRIGHT]" > footer.h2m
$BITCOIND --version | sed -n '1!p' >> footer.h2m
for cmd in $BITCOIND $BITCOINCLI $BITCOINTX $WALLET_TOOL $BITCOINUTIL $BITCOINQT; do
cmdname="${cmd##*/}"
help2man -N --version-string="${BTCVER[0]}" --include=footer.h2m -o "${MANDIR}/${cmdname}.1" "${cmd}"
sed -i "s/\\\-${BTCVER[1]}//g" "${MANDIR}/${cmdname}.1"
done
rm -f footer.h2m

@ -6,13 +6,14 @@ Release Process
### Before every release candidate
* Update translations see [translation_process.md](https://github.com/bitcoin/bitcoin/blob/master/doc/translation_process.md#synchronising-translations).
* Update manpages, see [gen-manpages.sh](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/README.md#gen-manpagessh).
* Update release candidate version in `configure.ac` (`CLIENT_VERSION_RC`).
* Update manpages (after rebuilding the binaries), see [gen-manpages.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/README.md#gen-manpagespy).
### Before every major and minor release
* Update [bips.md](bips.md) to account for changes since the last release (don't forget to bump the version number on the first line).
* Update version in `configure.ac` (don't forget to set `CLIENT_VERSION_RC` to `0`).
* Update manpages (see previous section)
* Write release notes (see "Write the release notes" below).
### Before every major release

Loading…
Cancel
Save