Possible bug for schema elements without targetNamespace

[apollo13]

I am trying to get a WSDL file from a 3rd party company to work. During parsing zeep fails to properly map some messages and therefore some operations (See details below). From what I can gather is that the main problem seems to be that the xsd:element for Add has no targetNamespace and when resolving zeep prefixes with the targetNamespace of wsdl:definitions:

python-zeep/src/zeep/wsdl/parse.py

Lines 36 to 43 in c391978

	tns = wsdl.target_namespace
	message_name = qname_attr(xmlelement, 'name', tns)
	parts = []
	for part in xmlelement.findall('wsdl:part', namespaces=NSMAP):
	part_name = part.get('name')
	part_element = qname_attr(part, 'element', tns)
	part_type = qname_attr(part, 'type', tns)

I am not sure if this is a bug in zeep or the WSDL, I can neither find a definitive answer in the XML Schema specs, nor in the WSDL ones. What I was able to find is from ws-i (http://www.ws-i.org/Profiles/BasicProfile-1.1.html#Schema_targetNamespace_Structure):

 R2105 All xsd:schema elements contained in a wsdl:types element of a DESCRIPTION MUST have a targetNamespace attribute with a valid and non-null value, UNLESS the xsd:schema element has xsd:import and/or xsd:annotation as its only child element(s). 

But I do not know if the targetNamespace from the wsdl:definitions should get inherited. I'll happily try to code a patch in zeep if you think this is a bug in zeep.

Version: zeep==3.1.0
WSDL: https://apolloner.eu/~apollo13/Calculator.wsdl
Example Script:

from lxml import etree
from zeep import Client

client = Client('https://apolloner.eu/~apollo13/Calculator.wsdl')

node = client.create_message(client.service, 'Add', [], {'a': 3, 'b': 4})

print(etree.tostring(node, pretty_print=True))

Output:

/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/wsdl/wsdl.py:338: UserWarning: The wsdl:message for '{http://Example.org}ICalculator_Add_InputMessage' contains an invalid part ('parameters'): invalid xsd type or elements
  warnings.warn(str(exc))
/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/wsdl/wsdl.py:338: UserWarning: The wsdl:message for '{http://Example.org}ICalculator_Add_OutputMessage' contains an invalid part ('parameters'): invalid xsd type or elements
  warnings.warn(str(exc))
/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/wsdl/definitions.py:130: UserWarning: The wsdl:operation 'Add' was not found in the wsdl:portType '{http://Example.org}ICalculator'
  warnings.warn(str(exc))
Forcing soap:address location to HTTPS
Traceback (most recent call last):
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/wsdl/definitions.py", line 149, in get
    return self._operations[key]
KeyError: 'Add'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/florian/test.py", line 8, in <module>
    client.service, 'Add', [], {'a': 3, 'b': 4})
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/client.py", line 131, in create_message
    operation_name, args, kwargs, client=self)
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/wsdl/bindings/soap.py", line 63, in _create
    operation_obj = self.get(operation)
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/wsdl/definitions.py", line 151, in get
    raise ValueError("No such operation %r on %s" % (key, self.name))
ValueError: No such operation 'Add' on {http://Example.org}DefaultBinding_ICalculator

[mvantellingen]

Could you check if the master branch works? I think the inheritance part of targetNamespace was indeed missing. See https://support.liquid-technologies.com/kb/a17/chameleon-schemas-schemas-without-a-target-namespace.aspx

Thanks for you work on Django btw 👍

[apollo13]

Hi @mvantellingen. Thank you very much. This does indeed fix the issue from the provided WSDL. I am running into another issue now and will prepare a new ticket later today.

Thank you very much for the quick response and the fix! Thanks for the Django compliment, but as you know Django is mostly a team effort :) That said, if you have a specific ticket there (I just realize you are working with oscar and wagtail) feel free to drop me a line and I'll see what I can do.

[apollo13]

@mvantellingen I am reopening this issue instead of creating a new one since I think it might boil down to the same issue.

New WSDL: http://apolloner.eu/~apollo13/Calculator2.wsdl
New Reproducer:

from lxml import etree
from zeep import Client

client = Client('https://apolloner.eu/~apollo13/Calculator2.wsdl')

node = client.create_message(client.service, 'Add', [], {'a': 3, 'b': 4})

print(etree.tostring(node, pretty_print=True))

results in

Traceback (most recent call last):
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/xsd/schema.py", line 251, in _get_instance
    for schema in self._get_schema_documents(qname.namespace):
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/xsd/schema.py", line 315, in _get_schema_documents
    return self.documents.get_by_namespace(namespace, fail_silently)
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/xsd/schema.py", line 349, in get_by_namespace
    "No schema available for the namespace %r" % namespace)
zeep.exceptions.NamespaceError: No schema available for the namespace None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/client.py", line 62, in __init__
    wsdl, self.transport, settings=self.settings)
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/wsdl/wsdl.py", line 84, in __init__
    root_definitions = Definition(self, document, self.location)
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/wsdl/wsdl.py", line 183, in __init__
    self.parse_types(doc)
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/wsdl/wsdl.py", line 325, in parse_types
    self.types.add_documents(schema_nodes, self.location, target_namespace)
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/xsd/schema.py", line 120, in add_documents
    document.resolve()
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/xsd/schema.py", line 470, in resolve
    _resolve_dict(self._elements)
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/xsd/schema.py", line 453, in _resolve_dict
    new = obj.resolve()
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/xsd/elements/element.py", line 255, in resolve
    self.resolve_type()
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/xsd/elements/element.py", line 252, in resolve_type
    self.type = self.type.resolve()
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/xsd/types/complex.py", line 333, in resolve
    self._element = self._element.resolve()
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/xsd/elements/indicators.py", line 212, in resolve
    self[i] = elm.resolve()
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/xsd/elements/references.py", line 22, in resolve
    elm = self._schema.get_element(self._ref)
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/xsd/schema.py", line 138, in get_element
    return self._get_instance(qname, 'get_element', 'element')
  File "/home/florian/dev/bap/bap-services/.venv/lib/python3.6/site-packages/zeep/xsd/schema.py", line 264, in _get_instance
    ) % (name, qname.text, qname.namespace))
zeep.exceptions.NamespaceError: Unable to resolve element demo. No schema available for the namespace None.

I am on zeep master now.

[apollo13]

Would you be okay with me sharing the original WSDL privately with you somehow? I rather not post it public without an okay from the other company.

[apollo13]

I have the beginnings of a patch in apollo13@488901a -- it sadly breaks two other tests; have to dig in (pointers appreciated :))

[apollo13]

The failing tests in question so you do not have to fetch my patch to see the failures:

============================================================================================== FAILURES ==============================================================================================
________________________________________________________________________________________ test_xml_empty_xmlns ________________________________________________________________________________________
Traceback (most recent call last):
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 580, in _get_component
    return items[qname]
KeyError: <lxml.etree.QName object at 0x7f4a7be8a648>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 453, in _resolve_dict
    new = obj.resolve()
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/elements/element.py", line 255, in resolve
    self.resolve_type()
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/elements/element.py", line 252, in resolve_type
    self.type = self.type.resolve()
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/types/complex.py", line 333, in resolve
    self._element = self._element.resolve()
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/elements/indicators.py", line 212, in resolve
    self[i] = elm.resolve()
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/elements/references.py", line 22, in resolve
    elm = self._schema.get_element(self._ref)
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 138, in get_element
    return self._get_instance(qname, 'get_element', 'element')
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 258, in _get_instance
    raise last_exception
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 254, in _get_instance
    return method(qname)
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 546, in get_element
    return self._get_component(qname, self._elements, 'element')
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 595, in _get_component
    location=self._location)
zeep.exceptions.LookupError: No element 'schema' in namespace http://tests.python-zeep.org/. Available elements are: {http://tests.python-zeep.org/}container

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/florian/sources/python-zeep/tests/test_xsd_integration.py", line 825, in test_xml_empty_xmlns
    """))
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 43, in __init__
    self.add_documents(nodes, location, target_namespace=tns)
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 120, in add_documents
    document.resolve()
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 470, in resolve
    _resolve_dict(self._elements)
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 465, in _resolve_dict
    'parent': obj.qname,
zeep.exceptions.LookupError: Unable to resolve element {http://tests.python-zeep.org/}schema in None. (via {http://tests.python-zeep.org/}container)
______________________________________________________________________________________ test_no_target_namespace ______________________________________________________________________________________
Traceback (most recent call last):
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 580, in _get_component
    return items[qname]
KeyError: <lxml.etree.QName object at 0x7f4a7bf0c328>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 453, in _resolve_dict
    new = obj.resolve()
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/elements/element.py", line 255, in resolve
    self.resolve_type()
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/elements/element.py", line 252, in resolve_type
    self.type = self.type.resolve()
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/types/complex.py", line 333, in resolve
    self._element = self._element.resolve()
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/elements/indicators.py", line 212, in resolve
    self[i] = elm.resolve()
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/elements/references.py", line 22, in resolve
    elm = self._schema.get_element(self._ref)
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 138, in get_element
    return self._get_instance(qname, 'get_element', 'element')
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 258, in _get_instance
    raise last_exception
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 254, in _get_instance
    return method(qname)
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 546, in get_element
    return self._get_component(qname, self._elements, 'element')
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 595, in _get_component
    location=self._location)
zeep.exceptions.LookupError: No element 'bla' in namespace http://tests.python-zeep.org/a. Available elements are: {http://tests.python-zeep.org/a}container

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/florian/sources/python-zeep/tests/test_xsd_schemas.py", line 594, in test_no_target_namespace
    xsd.Schema(node_a, transport=transport)
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 43, in __init__
    self.add_documents(nodes, location, target_namespace=tns)
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 120, in add_documents
    document.resolve()
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 470, in resolve
    _resolve_dict(self._elements)
  File "/home/florian/sources/python-zeep/.tox/py37/lib/python3.7/site-packages/zeep/xsd/schema.py", line 465, in _resolve_dict
    'parent': obj.qname,
zeep.exceptions.LookupError: Unable to resolve element {http://tests.python-zeep.org/a}bla in None. (via {http://tests.python-zeep.org/a}container)
========================================================================================== warnings summary ==========================================================================================
tests/test_asyncio_transport.py::test_cache_checks_type
  /home/florian/sources/python-zeep/tests/test_asyncio_transport.py:48: RuntimeWarning: coroutine 'test_cache_checks_type.<locals>.foo' was never awaited
    cache.add('x', foo())

tests/test_client.py::test_create_message
  /home/florian/sources/python-zeep/tests/test_client.py:126: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
    assert data

-- Docs: http://doc.pytest.org/en/latest/warnings.html
2 failed, 445 passed, 2 warnings in 2.56 seconds
ERROR: InvocationError for command '/home/florian/sources/python-zeep/.tox/py37/bin/coverage run --parallel -m pytest -q --tb=native' (exited with code 1)
______________________________________________________________________________________________ summary _______________________________________________________________________________________________
ERROR:   py37: commands failed

[mvantellingen]

Sure, you can mail the wsdl to michaelvantellingen@gmail.com

Regarding the failing unittest test_no_target_namespace. I did a quick fix on my end too and ran into the same failing unittest. It's actually basically the same scenario's as your xsd but it asserts the opposite result. I'm wondering if that unittest is actually ok.

[apollo13]

The one thing I was wondering is also if the fix is on the wrong level. Ie the test makes sense for xsd, but wsdl requires a bit of different rules maybe… But I am somewhat at a loss here with all the (non-)specs.

[mvantellingen]

Could you give master a try again? I think i found a good solution which hopefully doesn't break other wsdl's. (Although then we should add tests for those cases)