400 Bad Request, XML not as expected

[clnt]

Hi @CodeDredd,

I've been trying to implement this package today but have encountered a few issues along the way, such as exceptions about request body being empty when using withWsa(), now that I have removed that call I am getting a 400 Bad Request with everything I try even though the docs make it look very straightforward.

Here is a screenshot of my IDE, showing my class on the left creating the Soap client via your package and calling a method defined in the WSDL, and passing three parameters through with keys which I would expect to see in the XML request. However when I perform a dd() in the ExtSoapEncoder it gets passed through, the arguments are there but are not truly reflected in the generated XML.

For example it hasn't added the first parameter languageID at all, and it has added the startDate and endDate as param1, param2 etc not honouring the keys that I am passing through - why is this?

Also just to note I have defined the __call method on my class and applied trait when I took a look at one of the generated classes:

    /**
     * Execute soap call
     *
     * @param string $method
     * @param mixed $parameters
     * @return \CodeDredd\Soap\Client\Response|mixed
     */
    public function __call($method, $parameters)
    {
        if (static::hasMacro($method)) {
            return $this->macroCall($method, $parameters);
        }

        $validationClass = 'App\\Validations'
            . ucfirst(Str::camel($method))
            . 'Validation';
        if (class_exists($validationClass)) {
            $parameters = app()->call([$validationClass, 'validator'], ['parameters' => $parameters]);
        }

        return $this->soap->call($method, $parameters);
    }

I can generate a successful response from the API using the SoapUI application which generates the following XML for the request:

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:tem="http://tempuri.org/">
   <soap:Header/>
   <soap:Body>
      <tem:GetVisitSummariesByDate>
         <tem:languageID>1</tem:languageID>
         <tem:startDate>2021-04-10T00:00:00.000Z</tem:startDate>
         <tem:endDate>2021-04-14T00:00:00.000Z</tem:endDate>
      </tem:GetVisitSummariesByDate>
   </soap:Body>
</soap:Envelope>

The SoapUI configuration in terms of authentication is simply a Username and Password, WSS-Password Type set to PasswordText and the 'Add default wsa:To` option checked.

Am I missing something here or is this package not going to work the way I need it to? 🤔

Laravel 6, package version 1.5.1

Cheers,
Matt

[CodeDredd]

@clnt Heyho :)

if you have basic auth as authentication. try using:

Soap::withBasicAuth(username, password)->withWsa()->baseWsdl(...)

instead of withOptions. Have you tried it yet?

[clnt]

@CodeDredd Cheers for the quick response :)

I did try using withBasicAuth() before but ran into errors.

If I try that approach I hit the exception I was getting before where it hits the WsaMiddleware and then errors in the fromStream method shown in screenshot below:

[CodeDredd]

Ahhh so if i get it right your using WSSE. Have you tried that too ?

Soap::withWsse([
'user_token_name' => 'username',
'user_token_password' => 'password'
])->baseWsdl(...)

[clnt]

@CodeDredd I don't believe I am based upon the settings I am putting into SoapUI? Its just Username/Password and I specify PasswordText which seems like some form of generic unsecure login.

I have tried the code above but unfortunately I just get the same exception as above.

I have tried connecting to the client using the PHP SoapClient class directly today, when specifying the soap_version to SOAP_1_2 the request just runs and never completes (it also does this when using your package and specifying this option) which is the reason for the ini set line attempting to see if that gave me anything to go on:

        /**
         * PHP Native Test
         */
        ini_set('default_socket_timeout', 600);

        $client = new \SoapClient($wsdl, [
            'soap_version' => SOAP_1_2,
            'trace' => 1,
            'login' => $userName,
            'password' => $password,
        ]);

        $toHeader = new \SoapHeader('http://www.w3.org/2005/08/addressing', 'To', $wsdl);

        $client->__setSoapHeaders(['To' => $toHeader]);
        $client->__setLocation(str_replace('?wsdl', '', $wsdl));

        $parameters = [
            'languageID' => 1,
            'startDate' => '2021-04-10T00:00:00.000Z',
            'endDate' => '2021-04-14T00:00:00.000Z',
        ];

        $result = $client->__soapCall('GetVisitSummariesByDate', [$parameters]);

I don't know where the problem lies but right now I feel it could be the API I am interacting with...

However, I did attempt to test the below free SOAP service using your package which doesn't have any auth, getting the same issues of requests not completing when specifying the soap version to the client:

        $result = Soap::baseWsdl('https://www.w3schools.com/xml/tempconvert.asmx')
            ->withOptions(['soap_version' => SOAP_1_2])
            ->call('FahrenheitToCelsius', [
                'Fahrenheit' => 75
            ]);
        dd($result);

Also just to clarify the error I get when it eventually times out upon specifying the soap_version it is SoapFault : Error Fetching http headers

Here is the wsp:Policy section taken from the API's WSDL that I am trying to connect to, I hope this can help clarify auth requirements:

<wsp:Policy wsu:Id="WSHttpBinding_policy">
    <wsp:ExactlyOne>
        <wsp:All>
            <sp:TransportBinding xmlns:sp="http://schemas.xmlsoap.org/ws/2005/07/securitypolicy">
                <wsp:Policy>
                    <sp:TransportToken>
                        <wsp:Policy>
                            <sp:HttpsToken RequireClientCertificate="false" />
                        </wsp:Policy>
                    </sp:TransportToken>
                    <sp:AlgorithmSuite>
                        <wsp:Policy>
                            <sp:Basic256 />
                        </wsp:Policy>
                    </sp:AlgorithmSuite>
                    <sp:Layout>
                        <wsp:Policy>
                            <sp:Strict />
                        </wsp:Policy>
                    </sp:Layout>
                    <sp:IncludeTimestamp />
                </wsp:Policy>
            </sp:TransportBinding>
            <sp:SignedSupportingTokens xmlns:sp="http://schemas.xmlsoap.org/ws/2005/07/securitypolicy">
                <wsp:Policy>
                    <sp:UsernameToken
                        sp:IncludeToken="http://schemas.xmlsoap.org/ws/2005/07/securitypolicy/IncludeToken/AlwaysToRecipient">
                        <wsp:Policy>
                            <sp:WssUsernameToken10 />
                        </wsp:Policy>
                    </sp:UsernameToken>
                </wsp:Policy>
            </sp:SignedSupportingTokens>
            <sp:Wss11 xmlns:sp="http://schemas.xmlsoap.org/ws/2005/07/securitypolicy">
                <wsp:Policy />
            </sp:Wss11>
            <sp:Trust10 xmlns:sp="http://schemas.xmlsoap.org/ws/2005/07/securitypolicy">
                <wsp:Policy>
                    <sp:MustSupportIssuedTokens />
                    <sp:RequireClientEntropy />
                    <sp:RequireServerEntropy />
                </wsp:Policy>
            </sp:Trust10>
            <wsaw:UsingAddressing />
        </wsp:All>
    </wsp:ExactlyOne>
</wsp:Policy>

[CodeDredd]

@clnt Thanks for the many input.

First of all i tried to tackle the simple problem with the "Fahrenheit" request. And there were two problems i encountered. i also maybe need to catch HTML Responses from soap reuqests which i encountered there. After looking at the error message i also found out, that the wsdl from w3schools is "broken" ^^. It has a wrong location url.

Second i found a bug using the call method directly. I needed to wrap the arguments in an blank array because __soapCall needs that as you rightly used. (Made a release because of that)

So i finally made it work with that request (Your request example has also as error the missing "?wsdl"):

        // location has to be set because the wsdl has a wrong location declaration. 
        // Its still set to "http" instead of "https" ;)
        $client = Soap::baseWsdl('https://www.w3schools.com/xml/tempconvert.asmx?wsdl')
                      ->withOptions([
                          'soap_version' => SOAP_1_2,
                          'location' => 'https://www.w3schools.com/xml/tempconvert.asmx?wsdl',
                      ]);
        $result = $client->call('FahrenheitToCelsius', [
            'Fahrenheit' => 75,
        ]);
        self::assertArrayHasKey('FahrenheitToCelsiusResult', $result->json());

        $result = $client->FahrenheitToCelsius([
            'Fahrenheit' => 75,
        ]);
        self::assertArrayHasKey('FahrenheitToCelsiusResult', $result->json());

Now lets still tackle your auth problem. I am pretty sure its WSSE if i look at your policy. But i am not that an expert in soap to say that i am 100% sure there.
In WSSESoap.php there is this code and thats why i think that:

        if ($password) {
            $passType = self::WSUNAME.'#PasswordText';
            if ($passwordDigest) {
                $password = base64_encode(sha1($nonce.$createdate.$password, true));
                $passType = self::WSUNAME.'#PasswordDigest';
            }
            $passwordNode = $this->soapDoc->createElementNS(self::WSSENS,  self::WSSEPFX.':Password');
            $token->appendChild($passwordNode);
            $passwordText = $this->soapDoc->createTextNode($password);
            $passwordNode->appendChild($passwordText);
            $passwordNode->setAttribute('Type', $passType);
        }

Maybe you write me your skypename or discordname and we make a call to fix also your auth problem. But maybe i gave you already a right hint.

I also made an mistake. The correct WSSE usage is:

Soap::baseWsdl('http://test.com'/v1?wsdl)->withWsse([
        'userTokenName' => 'username',
        'userTokenPassword' => 'password',
    ])

[clnt]

@CodeDredd No problem thanks for your time taking a look into this!

I believe progress is being made, although currently I have to test your changes in a fresh Laravel 8 install as I am limited to using Laravel 6 currently for the project consuming the API which is the main reason I installed your package so if we could get any fixes over to support Laravel 6 that would be great.

I've tried again using master branch of this package in Laravel 8, adjusting the authentication like you said, also including the ?wsdl, and I am not getting any errors now it thinks the response is okay (also not getting an error about content-type header being incorrect due to not specifying soap version). However, the body is null (when I can get contents from API via SoapUI) - when I try to use SOAP_1_2 in the version I just get a test that keeps running until timeout again, but without it specified currently I am not getting an error.

What I am seeing though is the XML doesn't look like its generating correctly again, there is now the presence of the languageID key and value which is produced in valid XML, but the startDate and endDate are missing values and also closing tags as you will see in the screenshot below, this dd() is the $result in the test shown:

I've had a quick dig around in the package code this morning but not 100% sure on where its generating/parsing the request XML. I think if we can get all those parameters into that envelope as valid XML I believe it could work

[CodeDredd]

@clnt good to hear.

I think your Date problem is realted with Issue #89 . Try what they said. They use:
DateTime::createFromFormat('Y-m-d\TH:i:s', date('Y-m-d\TH:i:s', strtotime(now()))) as an example

[clnt]

@CodeDredd Thanks for that, that indeed solved the XML generation by overriding the typemaps.

There is unfortunately still some kind of issue I am not quite sure what is going on as I ended up going through various dependencies to try and track down where it was going wrong but essentially even when I was getting an error from the server for example incorrect headers, the package was returning a successful response with null as body because in the SoapClient class it was getting null as the result from the request in the call method.

When I've tried to specify these headers manually including the action header within the Content-Type for SOAP 1.2 I get a 400 Bad Request from server, however when I specify soap_version in the client options and specify SOAP_1_2 its just timing out all the time. Not really sure how to approach the soap_version error, I can dd() the contents of the $psr7Request on HttPlugHandle but when it sends the request on the next line that is when the timeout occurs.

[clnt]

@CodeDredd I am currently writing an implementation using the native php client, I have a feeling it could be down to the WS Security headers that are being generated (I assume they are, but I haven't seen it included in any dd() of the request that I have attempted when debugging).

Once I have it working I will see if I can do any further debugging on why this package seems to not be working for my use case. From doing plenty of reading online it seems like different SOAP APIs have their quirks..

[CodeDredd]

@clnt hmmm...thanks for keeping me informed. I am really like to know where it is failing. I also going to start at some point to make this api lighter.
SOAP is just here to make our lives a bit more suffering ^^

[clnt]

@CodeDredd I managed to get it working successfully last night with the following class after figuring out how to lift the SoapUI generated security header and also a fair bit of a searching!

<?php

namespace App\Integrations\Api;

use Illuminate\Support\Str;
use SoapHeader;
use SoapVar;

class ApiSoapClient extends \SoapClient
{
    /** @var string */
    private $oasis = 'http://docs.oasis-open.org/wss/2004/01';

    /** @var string */
    private $wsa = 'http://www.w3.org/2005/08/addressing';

    /** @var string */
    private $wsdl;

    /** @var string */
    private $username;

    /** @var string */
    private $password;

    /** @var string */
    private $passwordType;

    public function __construct(string $wsdl, array $options = [])
    {
        $this->wsdl = $wsdl;
        $this->username = config('api.user_name');
        $this->password = config('api.password');
        $this->passwordType = 'PasswordText';

        $options['soap_version'] = SOAP_1_2;
        $options['trace'] = 1;

        parent::__construct($wsdl, $options);
    }

    public function call(string $name, array $arguments)
    {
        try {
            return $this->__soapCall($name, [$arguments], [
                'location' => str_replace('?wsdl', '', $this->wsdl),
            ], [
                $this->generateWSSecurityHeader(),
                $this->generateWsaActionHeader($name),
                $this->generateWsaToHeader(),
            ]);
        } catch (\Exception $exception) {
            dd($this->__getLastRequest(), $exception);
        }
    }

    /** @return SoapHeader|string */
    protected function generateWSSecurityHeader()
    {
        $xml = '
        <wsse:Security env:mustUnderstand="1" xmlns:wsse="' . $this->oasis . '/oasis-200401-wss-wssecurity-secext-1.0.xsd" 
        ' . 'xmlns:wsu="'. $this->oasis . '/oasis-200401-wss-wssecurity-utility-1.0.xsd">' . ' 
        <wsse:UsernameToken wsu:Id="UsernameToken-' . Str::uuid() . '">
        <wsse:Username>' . $this->username . '</wsse:Username>
        <wsse:Password Type="' . $this->oasis . '/oasis-200401-wss-username-token-profile-1.0#' . $this->passwordType . '">' . $this->password . '</wsse:Password>
        <wsse:Nonce EncodingType="' . $this->oasis . '/oasis-200401-wss-soap-message-security-1.0#Base64Binary">' . sha1(mt_rand()) . '</wsse:Nonce>' . '
        <wsu:Created>' . now()->format('Y-m-d\TH:i:s.000\Z') . '</wsu:Created>' . '
        </wsse:UsernameToken>
        </wsse:Security>';

        return new SoapHeader(
            $this->oasis . '/oasis-200401-wss-wssecurity-secext-1.0.xsd',
            'Security',
            new SoapVar($xml, XSD_ANYXML),
            true
        );
    }

    protected function generateWsaActionHeader(string $action): SoapHeader
    {
        $xml = '<wsa:Action xmlns:wsa="' . $this->wsa . '">'
            . 'http://tempuri.org/IBEAPI/' . $action
            . '</wsa:Action>';

        return new SoapHeader(
            $this->wsa,
            'Action',
            new SoapVar($xml, XSD_ANYXML),
        );
    }

    protected function generateWsaToHeader(): SoapHeader
    {
        $xml = '<wsa:To xmlns:wsa="' . $this->wsa . '">'
            . str_replace('?wsdl', '', $this->wsdl)
            . '</wsa:To>';

        return new SoapHeader(
            $this->wsa,
            'To',
            new SoapVar($xml, XSD_ANYXML),
        );
    }
}

While writing this I figured its the env:mustUnderstand="1" that causes the php soap client to run until timeout.

Is it possible at all to pass down SoapHeader classes to your package to override headers? I feel like that could potentially give better control.

Yeah i totally agree! I feel like it shouldn't have taken the time it has just to get a successful response from the API, looking at peoples answers from 11 years ago! :D I hope the above can help you see where your package may be handling something differently, I am going to carry on with implementing the API based off my current client class so that things are moving however I am keen to switch back to using the package if possible due to better error handling, testing etc

[CodeDredd]

@clnt Thanks for the detail answer. Yeah the typical soap struggle. Well this soap api is extendable and you can change the headers by middlewares. I recently made one custom for DHL.
I try to get it working the way you need it. Actually you can debug the request with its whole XML so you can check if its right. See the unit test i made for example for DHL auth.

[CodeDredd]

@clnt ok i finally took the time. I made a PR. Please look into it if it works with that. Added a new option for the Wsse Heaader:

        $client = Soap::baseWsdl('https://laravel-soap.wsdl')->withWsse([
            'userTokenName' => 'Test',
            'userTokenPassword' => 'passwordTest',
            'mustUnderstand' => false
        ])->withWsa();

you can look at the unit test if its now ok. It would be awesome if you could test it :-)