Looking at #1028371, should generated dependencies on python3-protobuf be
  python3-protobuf (>= 3.21), python3-protobuf (<< 3.22)
to ensure that the binary package is used with the same version
as the protobuf-compiler used during the build?

If yes, are other language bindings also affected by this problem?

 You mean on python3-bernhard, right?
 Then what? It would become uninstallable when a new protobuf version
(3.22 next time) is uploaded and built.
 I still don't get your point. How does a language binding package
version relate to the protobuf-compiler version? I don't follow the
internals of Protobuf, but I would say it's more related to the
library soname and its provided API version.

Regards,
Laszlo/GCS

Hello,

yes

this is the idea, since right now even a basic import of bernhard is just not working
version is uploaded.
The idea is to restrict the protobuf version it was built with, to make sure the import works.

E.g. this is the current pb.py autogenerated code
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler.  DO NOT EDIT!
# source: riemann.proto

from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)

_sym_db = _symbol_database.Default()




DESCRIPTOR = _descriptor.FileDescriptor(
   name='riemann.proto',
   package='',
   syntax='proto2',
   serialized_options=b'\n\021com.aphyr.riemannB\005Proto',
   create_key=_descriptor._internal_create_key,
   serialized_pb=b'\n\rriemann.proto\"\x81\x01\n\x05State\x12\x0c\n\x04time\x18\x01 \x01(\x03\x12\r\n\x05state\x18\x02 \x01(\t\x12\x0f\n\x07service\x18\x03 \x01(\t\x12\x0c\n\x04host\x18\x04 \x01(\t\x12\x13\n\x0b\x64\x65scription\x18\x05 \x01(\t\x12\x0c\n\x04once\x18\x06 \x01(\x08\x12\x0c\n\x04tags\x18\x07 \x03(\t\x12\x0b\n\x03ttl\x18\x08 \x01(\x02\"\xce\x01\n\x05\x45vent\x12\x0c\n\x04time\x18\x01 \x01(\x03\x12\r\n\x05state\x18\x02 \x01(\t\x12\x0f\n\x07service\x18\x03 \x01(\t\x12\x0c\n\x04host\x18\x04 \x01(\t\x12\x13\n\x0b\x64\x65scription\x18\x05 \x01(\t\x12\x0c\n\x04tags\x18\x07 \x03(\t\x12\x0b\n\x03ttl\x18\x08 \x01(\x02\x12\x1e\n\nattributes\x18\t \x03(\x0b\x32\n.Attribute\x12\x15\n\rmetric_sint64\x18\r \x01(\x12\x12\x10\n\x08metric_d\x18\x0e \x01(\x01\x12\x10\n\x08metric_f\x18\x0f \x01(\x02\"\x17\n\x05Query\x12\x0e\n\x06string\x18\x01 \x01(\t\"g\n\x03Msg\x12\n\n\x02ok\x18\x02 \x01(\x08\x12\r\n\x05\x65rror\x18\x03 \x01(\t\x12\x16\n\x06states\x18\x04 \x03(\x0b\x32\x06.State\x12\x15\n\x05query\x18\x05 \x01(\x0b\x32\x06.Query\x12\x16\n\x06\x65vents\x18\x06 \x03(\x0b\x32\x06.Event\"\'\n\tAttribute\x12\x0b\n\x03key\x18\x01 \x02(\t\x12\r\n\x05value\x18\x02 \x01(\tB\x1a\n\x11\x63om.aphyr.riemannB\x05Proto'
)




_STATE = _descriptor.Descriptor(
   name='State',
   full_name='State',
   filename=None,
   file=DESCRIPTOR,
   containing_type=None,
   create_key=_descriptor._internal_create_key,
   fields=[
     _descriptor.FieldDescriptor(
       name='time', full_name='State.time', index=0,
       number=1, type=3, cpp_type=2, label=1,
       has_default_value=False, default_value=0,
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='state', full_name='State.state', index=1,
       number=2, type=9, cpp_type=9, label=1,
       has_default_value=False, default_value=b"".decode('utf-8'),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='service', full_name='State.service', index=2,
       number=3, type=9, cpp_type=9, label=1,
       has_default_value=False, default_value=b"".decode('utf-8'),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='host', full_name='State.host', index=3,
       number=4, type=9, cpp_type=9, label=1,
       has_default_value=False, default_value=b"".decode('utf-8'),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='description', full_name='State.description', index=4,
       number=5, type=9, cpp_type=9, label=1,
       has_default_value=False, default_value=b"".decode('utf-8'),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='once', full_name='State.once', index=5,
       number=6, type=8, cpp_type=7, label=1,
       has_default_value=False, default_value=False,
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='tags', full_name='State.tags', index=6,
       number=7, type=9, cpp_type=9, label=3,
       has_default_value=False, default_value=[],
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='ttl', full_name='State.ttl', index=7,
       number=8, type=2, cpp_type=6, label=1,
       has_default_value=False, default_value=float(0),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
   ],
   extensions=[
   ],
   nested_types=[],
   enum_types=[
   ],
   serialized_options=None,
   is_extendable=False,
   syntax='proto2',
   extension_ranges=[],
   oneofs=[
   ],
   serialized_start=18,
   serialized_end=147,
)


_EVENT = _descriptor.Descriptor(
   name='Event',
   full_name='Event',
   filename=None,
   file=DESCRIPTOR,
   containing_type=None,
   create_key=_descriptor._internal_create_key,
   fields=[
     _descriptor.FieldDescriptor(
       name='time', full_name='Event.time', index=0,
       number=1, type=3, cpp_type=2, label=1,
       has_default_value=False, default_value=0,
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='state', full_name='Event.state', index=1,
       number=2, type=9, cpp_type=9, label=1,
       has_default_value=False, default_value=b"".decode('utf-8'),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='service', full_name='Event.service', index=2,
       number=3, type=9, cpp_type=9, label=1,
       has_default_value=False, default_value=b"".decode('utf-8'),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='host', full_name='Event.host', index=3,
       number=4, type=9, cpp_type=9, label=1,
       has_default_value=False, default_value=b"".decode('utf-8'),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='description', full_name='Event.description', index=4,
       number=5, type=9, cpp_type=9, label=1,
       has_default_value=False, default_value=b"".decode('utf-8'),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='tags', full_name='Event.tags', index=5,
       number=7, type=9, cpp_type=9, label=3,
       has_default_value=False, default_value=[],
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='ttl', full_name='Event.ttl', index=6,
       number=8, type=2, cpp_type=6, label=1,
       has_default_value=False, default_value=float(0),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='attributes', full_name='Event.attributes', index=7,
       number=9, type=11, cpp_type=10, label=3,
       has_default_value=False, default_value=[],
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='metric_sint64', full_name='Event.metric_sint64', index=8,
       number=13, type=18, cpp_type=2, label=1,
       has_default_value=False, default_value=0,
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='metric_d', full_name='Event.metric_d', index=9,
       number=14, type=1, cpp_type=5, label=1,
       has_default_value=False, default_value=float(0),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='metric_f', full_name='Event.metric_f', index=10,
       number=15, type=2, cpp_type=6, label=1,
       has_default_value=False, default_value=float(0),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
   ],
   extensions=[
   ],
   nested_types=[],
   enum_types=[
   ],
   serialized_options=None,
   is_extendable=False,
   syntax='proto2',
   extension_ranges=[],
   oneofs=[
   ],
   serialized_start=150,
   serialized_end=356,
)


_QUERY = _descriptor.Descriptor(
   name='Query',
   full_name='Query',
   filename=None,
   file=DESCRIPTOR,
   containing_type=None,
   create_key=_descriptor._internal_create_key,
   fields=[
     _descriptor.FieldDescriptor(
       name='string', full_name='Query.string', index=0,
       number=1, type=9, cpp_type=9, label=1,
       has_default_value=False, default_value=b"".decode('utf-8'),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
   ],
   extensions=[
   ],
   nested_types=[],
   enum_types=[
   ],
   serialized_options=None,
   is_extendable=False,
   syntax='proto2',
   extension_ranges=[],
   oneofs=[
   ],
   serialized_start=358,
   serialized_end=381,
)


_MSG = _descriptor.Descriptor(
   name='Msg',
   full_name='Msg',
   filename=None,
   file=DESCRIPTOR,
   containing_type=None,
   create_key=_descriptor._internal_create_key,
   fields=[
     _descriptor.FieldDescriptor(
       name='ok', full_name='Msg.ok', index=0,
       number=2, type=8, cpp_type=7, label=1,
       has_default_value=False, default_value=False,
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='error', full_name='Msg.error', index=1,
       number=3, type=9, cpp_type=9, label=1,
       has_default_value=False, default_value=b"".decode('utf-8'),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='states', full_name='Msg.states', index=2,
       number=4, type=11, cpp_type=10, label=3,
       has_default_value=False, default_value=[],
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='query', full_name='Msg.query', index=3,
       number=5, type=11, cpp_type=10, label=1,
       has_default_value=False, default_value=None,
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='events', full_name='Msg.events', index=4,
       number=6, type=11, cpp_type=10, label=3,
       has_default_value=False, default_value=[],
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
   ],
   extensions=[
   ],
   nested_types=[],
   enum_types=[
   ],
   serialized_options=None,
   is_extendable=False,
   syntax='proto2',
   extension_ranges=[],
   oneofs=[
   ],
   serialized_start=383,
   serialized_end=486,
)


_ATTRIBUTE = _descriptor.Descriptor(
   name='Attribute',
   full_name='Attribute',
   filename=None,
   file=DESCRIPTOR,
   containing_type=None,
   create_key=_descriptor._internal_create_key,
   fields=[
     _descriptor.FieldDescriptor(
       name='key', full_name='Attribute.key', index=0,
       number=1, type=9, cpp_type=9, label=2,
       has_default_value=False, default_value=b"".decode('utf-8'),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
     _descriptor.FieldDescriptor(
       name='value', full_name='Attribute.value', index=1,
       number=2, type=9, cpp_type=9, label=1,
       has_default_value=False, default_value=b"".decode('utf-8'),
       message_type=None, enum_type=None, containing_type=None,
       is_extension=False, extension_scope=None,
       serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
   ],
   extensions=[
   ],
   nested_types=[],
   enum_types=[
   ],
   serialized_options=None,
   is_extendable=False,
   syntax='proto2',
   extension_ranges=[],
   oneofs=[
   ],
   serialized_start=488,
   serialized_end=527,
)

_EVENT.fields_by_name['attributes'].message_type = _ATTRIBUTE
_MSG.fields_by_name['states'].message_type = _STATE
_MSG.fields_by_name['query'].message_type = _QUERY
_MSG.fields_by_name['events'].message_type = _EVENT
DESCRIPTOR.message_types_by_name['State'] = _STATE
DESCRIPTOR.message_types_by_name['Event'] = _EVENT
DESCRIPTOR.message_types_by_name['Query'] = _QUERY
DESCRIPTOR.message_types_by_name['Msg'] = _MSG
DESCRIPTOR.message_types_by_name['Attribute'] = _ATTRIBUTE
_sym_db.RegisterFileDescriptor(DESCRIPTOR)

State = _reflection.GeneratedProtocolMessageType('State', (_message.Message,), {
   'DESCRIPTOR' : _STATE,
   '__module__' : 'riemann_pb2'
   # @@protoc_insertion_point(class_scope:State)
   })
_sym_db.RegisterMessage(State)

Event = _reflection.GeneratedProtocolMessageType('Event', (_message.Message,), {
   'DESCRIPTOR' : _EVENT,
   '__module__' : 'riemann_pb2'
   # @@protoc_insertion_point(class_scope:Event)
   })
_sym_db.RegisterMessage(Event)

Query = _reflection.GeneratedProtocolMessageType('Query', (_message.Message,), {
   'DESCRIPTOR' : _QUERY,
   '__module__' : 'riemann_pb2'
   # @@protoc_insertion_point(class_scope:Query)
   })
_sym_db.RegisterMessage(Query)

Msg = _reflection.GeneratedProtocolMessageType('Msg', (_message.Message,), {
   'DESCRIPTOR' : _MSG,
   '__module__' : 'riemann_pb2'
   # @@protoc_insertion_point(class_scope:Msg)
   })
_sym_db.RegisterMessage(Msg)

Attribute = _reflection.GeneratedProtocolMessageType('Attribute', (_message.Message,), {
   'DESCRIPTOR' : _ATTRIBUTE,
   '__module__' : 'riemann_pb2'
   # @@protoc_insertion_point(class_scope:Attribute)
   })
_sym_db.RegisterMessage(Attribute)


DESCRIPTOR._options = None
# @@protoc_insertion_point(module_scope)


this is failing to load
python3 -c "import bernhard"
Traceback (most recent call last):
   File "<string>", line 1, in <module>
   File "/usr/lib/python3/dist-packages/bernhard/__init__.py", line 20, in <module>
     from . import pb
   File "/usr/lib/python3/dist-packages/bernhard/pb.py", line 36, in <module>
     _descriptor.FieldDescriptor(
   File "/usr/lib/python3/dist-packages/google/protobuf/descriptor.py", line 560, in __new__
     _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors should not be created directly, but only retrieved from their parent.



This is the new pb.py built with newer protobuf
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler.  DO NOT EDIT!
# source: riemann.proto
"""Generated protocol buffer code."""
from google.protobuf.internal import builder as _builder
from google.protobuf import descriptor as _descriptor
from google.protobuf import descriptor_pool as _descriptor_pool
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)

_sym_db = _symbol_database.Default()




DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\rriemann.proto\"\x81\x01\n\x05State\x12\x0c\n\x04time\x18\x01 \x01(\x03\x12\r\n\x05state\x18\x02 \x01(\t\x12\x0f\n\x07service\x18\x03 \x01(\t\x12\x0c\n\x04host\x18\x04 \x01(\t\x12\x13\n\x0b\x64\x65scription\x18\x05 \x01(\t\x12\x0c\n\x04once\x18\x06 \x01(\x08\x12\x0c\n\x04tags\x18\x07 \x03(\t\x12\x0b\n\x03ttl\x18\x08 \x01(\x02\"\xce\x01\n\x05\x45vent\x12\x0c\n\x04time\x18\x01 \x01(\x03\x12\r\n\x05state\x18\x02 \x01(\t\x12\x0f\n\x07service\x18\x03 \x01(\t\x12\x0c\n\x04host\x18\x04 \x01(\t\x12\x13\n\x0b\x64\x65scription\x18\x05 \x01(\t\x12\x0c\n\x04tags\x18\x07 \x03(\t\x12\x0b\n\x03ttl\x18\x08 \x01(\x02\x12\x1e\n\nattributes\x18\t \x03(\x0b\x32\n.Attribute\x12\x15\n\rmetric_sint64\x18\r \x01(\x12\x12\x10\n\x08metric_d\x18\x0e \x01(\x01\x12\x10\n\x08metric_f\x18\x0f \x01(\x02\"\x17\n\x05Query\x12\x0e\n\x06string\x18\x01 \x01(\t\"g\n\x03Msg\x12\n\n\x02ok\x18\x02 \x01(\x08\x12\r\n\x05\x65rror\x18\x03 \x01(\t\x12\x16\n\x06states\x18\x04 \x03(\x0b\x32\x06.State\x12\x15\n\x05query\x18\x05 \x01(\x0b\x32\x06.Query\x12\x16\n\x06\x65vents\x18\x06 \x03(\x0b\x32\x06.Event\"\'\n\tAttribute\x12\x0b\n\x03key\x18\x01 \x02(\t\x12\r\n\x05value\x18\x02 \x01(\tB\x1a\n\x11\x63om.aphyr.riemannB\x05Proto')

_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, globals())
_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'riemann_pb2', globals())
if _descriptor._USE_C_DESCRIPTORS == False:

   DESCRIPTOR._options = None
   DESCRIPTOR._serialized_options = b'\n\021com.aphyr.riemannB\005Proto'
   _STATE._serialized_start=18
   _STATE._serialized_end=147
   _EVENT._serialized_start=150
   _EVENT._serialized_end=356
   _QUERY._serialized_start=358
   _QUERY._serialized_end=381
   _MSG._serialized_start=383
   _MSG._serialized_end=486
   _ATTRIBUTE._serialized_start=488
   _ATTRIBUTE._serialized_end=527
# @@protoc_insertion_point(module_scope)



And this is how the files are generated:
override_dh_auto_build:
         protoc -I=debian --python_out=bernhard debian/riemann.proto
         mv bernhard/riemann_pb2.py bernhard/pb.py
         dh_auto_build


So, reverse-dependencies of protobuf, using protoc directly should probably have a strict dependency on protobuf runtime version.

G.

Yes, sorry.

That's the point - an uninstallable package is better than a broken
package, especially when being uninstallability blocks upgrading
python3-protobuf for users where it is known to cause breakage.

And it ensures that the package will be rebuilt (or removed)
during protobuf transitions.

$ python3 -c "import bernhard"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/bernhard/__init__.py", line 20, in <module>
    from . import pb
  File "/usr/lib/python3/dist-packages/bernhard/pb.py", line 36, in <module>
    _descriptor.FieldDescriptor(
  File "/usr/lib/python3/dist-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors should not be created directly, but only retrieved from their parent.
$ head -2 /usr/lib/python3/dist-packages/bernhard/pb.py
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler.  DO NOT EDIT!
$

cu
Adrian

Hi,

the problem in #1031716/#1028371 is as follows:

1. python3-protobuf and protobuf-compiler are both built from
   src:protobuf

2. packages using python3-protobuf (e.g. python3-bernhard) ship files
   that were generated with protobuf-compiler during their build
   (e.g. /usr/lib/python3/dist-packages/bernhard/pb.py )

3. The generated files in python3-protobuf rdeps like python3-bernhard
   might not work with python3-protobuf from a different major release
   of src:protobuf

The manual solution for packages like python3-bernhard would be
Build-Depends: protobuf-compiler (>= 3.21), protobuf-compiler (<< 3.22),
               python3-protobuf (>= 3.21), python3-protobuf (<< 3.22),
Depends: python3-protobuf (>= 3.21), python3-protobuf (<< 3.22),

Creating an own debhelper addon similar to dh_numpy3 would work for
automatically creating such dependencies, but is there any way for
python3-protobuf to inject such versioned runtime dependencies into
python3:Depends that does not require every rdep to manually add
addon usage?

Thanks
Adrian

Hi Adrian (2023.03.07_10:09:35_+0000)
They are consulted when translating Python .dist-info/.egg-info requires
into Debian dependencies.

You can see a (more complex) example in cffi:
https://salsa.debian.org/python-team/packages/python-cffi/-/blob/master/debian/gen-backend-versions.py

SR

On Tue, 7 Mar 2023 12:09:35 +0200 Adrian Bunk <bunk@debian.org> wrote:
 > 2. packages using python3-protobuf (e.g. python3-bernhard) ship files
 >    that were generated with protobuf-compiler during their build
 >    (e.g. /usr/lib/python3/dist-packages/bernhard/pb.py )

Can this generation be moved to postinst and use a dpkg trigger to
regenerate it when protobuf is updated?

An example is
https://sources.debian.org/src/node-constants-browserify/1.0.0%2Bdfsg-8/debian/postinst/

Though it would mean runtime dependency on protobuf-compiler.

this, given its RC severity.

With my still somewhat limited understanding, a strict version alignment
between protobuf-compiler and python3-protobuf would probably resolve
this particular symptom, but the issues here seem to run deeper.

Specifically:
  * The protobuf project provides three different versions of Python
    bindings: pure Python, C++, and libupb-based[1]. These are
    selectable using the PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION
    environment variable.

  * Debian's python3-protobuf, from src:protobuf, ships the pure Python
    version, as well as the C++ bindings. The default implementation in
    Debian is "cpp".

  * The upb implementation is not included in src:protobuf, but in the
    upb upstream source[2], i.e. what is src:upb in Debian, even though
    the snapshot we have in Debian does not contain sources to Python
    bindings.

  * Upstream has switched the default implementation to "upb", and
    deprecated the "cpp" implementation. There is, in fact, no way for
    one to fetch the "cpp" version from PyPI. This is documented
    extensively in their May 2022 release notes[3]. However, Debian
    still ships, and defaults to, cpp, a major departure from upstream.

  * Relatedly, when they made that switch, they also made changes to
    their versioning scheme, disconnecting the Python library's version
    from the source version. As a result, the Python API (both upb, as
    well as pure Python), is now versioned at "4.21", rather than
    "3.21". The Debian binary package python3-protobuf is versioned as
    "3.21.12-1", which is not a version that exists, or will ever exist,
    upstream. That binary package in fact, is shipping an egg named
    protobuf-4.21.12.egg-info. (This is all also well documented in their
    release notes[3]).

  * Finally, in the same release notes document[3], they also state:
    "Python upb requires generated code that has been generated from
    protoc 3.19.0 or newer.".

    Indeed, if one fetches protobuf 4.21 from PyPI, and runs:
      python3 -c 'import bernhard'
        or
      PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=upb python3 -c 'import bernhard'

    ...a traceback message is emitted, but a much more informative one:
    > TypeError: Descriptors cannot not be created directly.
    >
    > If this call came from a _pb2.py file, your generated code is out of
    > date and must be regenerated with protoc >= 3.19.0.
    >
    > If you cannot immediately regenerate your protos, some other possible workarounds are:
    > 1. Downgrade the protobuf package to 3.20.x or lower.
    > 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will
    > use pure-Python parsing and will be much slower).
    >
    > More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

  * The release notes specifically mention "upb" requiring protoc
    (protobuf-compiler) >= 3.19, but not "cpp". However, as established
    above, "cpp" is deprecated and not used by anyone (but Debian), and
    therefore they either meant "the non-Pure-Python implementation"
    there, or did not pay as much attention to forward- and
    backwards-compatibility, or informative error messages for their
    deprecated backend. It's likely, but not entirely clear, that the
    protoc dependency requirement is >= 3.19 here as well.

  * Finally note that the 3.21.12-1+b2 Python implementation still works
    with python3-bernhard, Built-Using:  protobuf-compiler (= 3.12.4-1+b3):
      PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python python3 -c 'import bernhard'

All in all: it's almost certainly necessary to make the dependency
tighter, to something like >= 3.19, if not tight to = 3.21.

I still feel uneasy about Debian shipping a version of python3-protobuf
that includes, and defaults to, an implementation that is deprecated
upstream (and on top of it, is misversioned). I'm not sure what to make
of this so late in the release cycle, though.

For trixie the path forward is probably something along the lines of
updating src:upb to a newer upstream, building the upb-based extension
as python3-protobuf-upb, and then changing src:protobuf to not build the
cpp extension, make python3-protobuf Arch: all, and then Recommend (or
Depend) on python3-protobuf-upb as the native/fast implementation.

Faidon

1: https://github.com/protocolbuffers/protobuf/tree/main/python
2: https://github.com/protocolbuffers/upb/tree/main/python
3: https://protobuf.dev/news/2022-05-06/

Hi,

On Mon, 27 Mar 2023 15:31:33 +0530 Pirate Praveen <praveen@onenetbeyond.org> wrote:
I think it could, but that's not really how the python ecosystem works
in Debian. It's also a solution for src:${reverse-dependency} (e.g.
closed bug #1028371) and not for this bug in src:protobuf.

I've been thinking a bit. While it would for sure be extremely nice if
src:protobuf would do this for its reverse build dependencies (ideally
only under the required conditions as I understand it's not always
needed), I think that the task to have the right (versioned)
dependencies is up to the reverse dependencies. Hence I think this bug
is important, but not serious.

Paul