Hello,

python3-debian is not able anymore to parse correctly some debian/control.in.
The first version with this issue is 0.1.44, so I suppose it is related to the
new RTS parser. The consequence of this bug is wrap-and-sort crashes when it
run on these files. Below is the error message:

Traceback (most recent call last):
  File "/usr/bin/wrap-and-sort", line 496, in <module>
    main()
  File "/usr/bin/wrap-and-sort", line 481, in main
    modified_files = wrap_and_sort(args)
                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/bin/wrap-and-sort", line 312, in wrap_and_sort
    control = WrapAndSortControl(control_file, args)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/bin/wrap-and-sort", line 99, in __init__
    super().__init__(filename, use_rts_parser=args.rts_parser)
  File "/usr/lib/python3/dist-packages/devscripts/control.py", line
210, in __init__
    self._deb822_file = parse_deb822_file(sequence)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/debian/_deb822_repro/parsing.py",
line 3095, in parse_deb822_file
    deb822_file = Deb822FileElement(LinkedList(tokens))
                                    ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/debian/_util.py", line 159, in __init__
    self.extend(values)
  File "/usr/lib/python3/dist-packages/debian/_util.py", line 272, in extend
    for v in values:
  File "/usr/lib/python3/dist-packages/debian/_deb822_repro/_util.py",
line 104, in _impl
    for token in token_stream:
  File "/usr/lib/python3/dist-packages/debian/_deb822_repro/_util.py",
line 104, in _impl
    for token in token_stream:
  File "/usr/lib/python3/dist-packages/debian/_deb822_repro/parsing.py",
line 2991, in _build_field_with_value
    value_element = next(buffered_stream, None)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/debian/_deb822_repro/_util.py",
line 143, in __next__
    return next(self._stream)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/debian/_deb822_repro/_util.py",
line 104, in _impl
    for token in token_stream:
  File "/usr/lib/python3/dist-packages/debian/_deb822_repro/parsing.py",
line 2939, in _build_value_line
    tokens_in_value = list(buffered_stream.takewhile(_non_end_of_line_token))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/debian/_deb822_repro/_util.py",
line 149, in takewhile
    while buffer or self._fill_buffer(5):
                    ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/debian/_deb822_repro/_util.py",
line 198, in _fill_buffer
    self._buffer.append(next(self._stream))
                        ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/debian/_deb822_repro/_util.py",
line 104, in _impl
    for token in token_stream:
  File "/usr/lib/python3/dist-packages/debian/_deb822_repro/parsing.py",
line 3031, in _abort_on_error_tokens
    raise SyntaxOrParseError(
debian._deb822_repro.types.SyntaxOrParseError: Syntax or Parse error
on the line: "%if USE_SYSTEM_ZLIB\n"

An easy way to reproduce the crash is to run wrap-and-sort on the
debian/ folder of
firefox-esr:

Best regards,
Dylan

Thanks for the bug report.

FWIW Deb822 isn't really built to support parsing deb822 files with
arbitrary data in the middle. This means it doesn't do well with at
least some .in files (which use a variety of styles).

That said, it would be good to handle this particular case better.

Cheers,

Jelmer

Thanks for the bug report.

FWIW Deb822 isn't really built to support parsing deb822 files with
arbitrary data in the middle. This means it doesn't do well with at
least some .in files (which use a variety of styles).

That said, it would be good to handle this particular case better.

Cheers,

Jelmer

Hi,

The RTS parser can already cope with non-deb822 input. However, the
caller has to opt-in to it and wrap-and-sort never that that.

I am doing a patch for this to have wrap-and-sort cope with invalid
syntactical input.  It will come with degradation of features - notably,
the package sorting will be disabled as otherwise we risk moving a
paragraph out of a templated section (and if that templated section is a
for-loop, you now get a completely different result).

To my knowledge, there is nothing for `python3-debian` to do at the
moment. Reassigning to devscripts for doing the patch.

Best regards,
Niels