[Mono-dev] HTTPS - MS .NET Client - Linux Mono Server - Interoperability

Wed Sep 28 05:42:42 EDT 2005

Hello Sebastien,

Things seems to go better for me, but I still have problems 
with my server side (Mono) authentication and decryption.
I now use a PKCS#12 file on my server side.
Se below for more info.

On Tue, 2005-09-27 at 14:47, Sebastien Pouliot wrote:
> Hello Yngve,
> 
> On Tue, 2005-27-09 at 12:01 +0200, Yngve Zackrisson wrote:
> > On Mon, 2005-09-26 at 18:10, Sebastien Pouliot wrote:
> > > Hello Yngve,
> > > 
> > > On Mon, 2005-26-09 at 16:26 +0200, Yngve Zackrisson wrote:
> > > > Hi.
> > > > 
> > > > I (like Martin Hinks) have problems with the interoperability 
> > > 
> > > Martin wasn't talking about HTTPS/SSL/TLS in it's latest email (unless I
> > > missed one). The subject was about password based key derivation.
> > > 
> > Martin wrote (snippet):
> > 
> > "Which .NET Encryption classes does Mono support that will work cross
> > platform? eg. I encrypt on Windows using .NET and the Linux class can
> > decrypt the same file and visa versa."
> > 
> > As I understand, he is asking what (symmetric?) encryption / decryption 
> > algorithms that is supported by both platforms. Right?. 
> 
> yes, your problems are about interop but not related to encryption
> (formats PVK/PKCS12 and certificate stores ;-).
> 
> > > > between a MS .NET HTTPS client and a Linux (Fedora Core 3) Mono server.
> > > > What I like to do is a HTTPS call from my MS .NET client to 
> > > > the Linux Mono server.
> > > > 
> > > > 
> > > > TEST1: mutual
> > > > -------------
> > > > 
> > > > My first test is based on the Mono mutual authentication example:
> > > > 
> > > > http://svn.myrealbox.com/source/trunk/mcs/class/Mono.Security/Test/tools/mutual/
> > > > 
> > > > The difference is that I create the certificates with openSSL 
> > > > and run the client on Win32 and MS .NET.
> > > > 
> > > > On my client I run: 
> > > > 
> > > > $ mutual.exe me292 TLS client16-cert.p12 xxxxxx
> > > > 
> > > > (where me292 is my server DNS and xxxxxx is the p12 password)
> > > > 
> > > > I then get: 
> > > > 
> > > > ...
> > > > CertificateValidation
> > > > CERTIFICATE: 
> > > >     ....
> > > > 
> > > >     Error(s)
> > > >         #-2146762487 
> > > > 
> > > > Meaning untrusted root, for my server certificate.
> > > > I have created a trusted CA cert using openssl x509 with the -trustout
> > > > option. What is the problem?. Must I have the .p12 in the Windows store?
> > > 
> > > If you're using Mono.Security (like the mutual test program) then you
> > > must install the root certificate into the *Mono* certificate store
> > > (even if you're executing the sample with the MS runtime).
> > > 
> > 
> > OK. I will try to do that ... 
> > (I will try to read in on what to install ... )
> > 
> > > You can also use MS' HttpWebRequest for mutual authentication. In this
> > > case (MS runtime + MS HttpWebRequest) you'll need to install the root
> > > certificate into the *Windows* certificate store.
> > 
> > or that ... .
> > 
> > > > After the printout of:
> > > > ...
> > > > PrivateKeySelection 
> > > >     ...
> > > > 
> > > >     PrivateKeySelection(1) 
> > > > 
> > > > no more happens on the client side.
> > > 
> > > That test program wasn't made to output anything useful. It only sends
> > > an "Hello" string to the server. Keep in mind that 'mutual', like
> > > others, is a *test* tool and not sample code.
> > > 
> > > > On the server side I run:
> > > > 
> > > > $ openssl s_server -www -cert server16-cert.pem -key server16-key.pem
> > > > -verify client16-cert.pem -CAfile cacert16t.pem
> > > > 
> > > > and get: 
> > > > 
> > > > verify depth is 0
> > > > Using default temp DH parameters
> > > > Using default temp ECDH parameters
> > > > ACCEPT
> > > > bad gethostbyaddr
> > > > depth=1 /C=SE/L=Stockholm/O=Test Company AB/OU=IT Department -
> > > > CA/CN=me292
> > > > validity return:1
> > > > depth=0 /C=SE/L=Sundsvall/O=Customer Company 16 AB/CN=Mirsad
> > > > validity return:1
> > > > 
> > > > 
> > > > AFAIK, A "Hello" should be written on the server.
> > > 
> > > An "Hello" is sent to the server (so the handshake starts) but it
> > > doesn't mean the server will output it on the console.
> > > 
> > > > 
> > > > TEST2: server
> > > > -------------
> > > > 
> > > > I have also made test with the Mono server example.
> > > > 
> > > > http://svn.myrealbox.com/source/trunk/mcs/class/Mono.Security/Test/tools/server/
> > > > 
> > > > Here I have had problems generating the private key of type .pvk.
> > > > In openssl there is an new option: 
> > > > 
> > > > openssl rsa -in server16-key.pem -pvk-strong -out
> > > > server16-key.strong.pvk
> > > >   "				 -pvk-weak    "	
> > > >   "				 -pvk-none    "
> > > > 
> > > > And I have tried all of them, But none have helped.
> > > 
> > > I never used OpenSSL to generate PVK files. Either use makecert to
> > > generate your certificates or look at my blog to use the PKCS#12 version
> > > of the server test tool.
> > 
> > OK. I will look at your blog and try to skip the PVK format 
> > in favor for PKCS#12.
> > 
> > > 
> > > > On the Mono server I run: 
> > > > 
> > > > $ mono --debug --verbose MonoSslHttpServer.exe 
> > > >  
> > > > I get a : 
> > > > 
> > > > System.Security.Cryptography.CryptographicException 
> > > > 
> > > > on the Mono server side in : 
> > > > 
> > > > (Mono.Security.Protocol.Tls.Handshake.HandshakeMessage:.ctor (...)
> > > > Mono.Security.Protocol.Tls.Handshake.Server.TlsClientCertificate.ProcessAsTls1 () 
> > > > 
> > > > Can this exception occure due to that the created pvk-file format 
> > > > is not compatible with the makecert equalent?
> > > 
> > > Maybe. PVK is a (bad) Microsoft format (mostly) used for Authenticode.
> > > The only interop testing has been done wrt to MS version of makecert. 
> > > 
> > > I strongly suggest you not to use PVK in real-life applications (weak
> > > encryption).
> > > 
> > > > Or can this be a result of some other thing?.
> > > > 
> > > > 
> > > > On the client side I now run on Linux the command: 
> > > > 
> > > > $ openssl s_client -connect 192.168.0.246:4433 -key
> > > > private/client16u-key.pem -cert client16u-cert.pem -CAfile cacert16t.pem
> > > > -state -msg
> > > > 
> > > > An I get the result (only the last part extracted): 
> > > > 
> > > > ...
> > > > SSL_connect:SSLv3 write client key exchange A
> > > > >>> TLS 1.0 Handshake [length 0046], CertificateVerify
> > > >     0f 00 00 42 00 40 3c de b4 0f 64 47 38 d3 ce e5
> > > >     f8 24 58 de 2c b1 25 23 4d 9a dd b8 e1 56 de 33
> > > >     20 6f c7 86 c6 a0 08 b9 a4 47 45 62 99 47 d0 2a
> > > >     36 b2 76 1e 54 30 20 09 e6 6e 36 96 4c 2d b9 a5
> > > >     2a 80 20 a5 87 3d
> > > > SSL_connect:SSLv3 write certificate verify A
> > > > >>> TLS 1.0 ChangeCipherSpec [length 0001]
> > > >     01
> > > > SSL_connect:SSLv3 write change cipher spec A
> > > > >>> TLS 1.0 Handshake [length 0010], Finished
> > > >     14 00 00 0c 18 1c 7c a3 bf 98 d8 71 fb a3 cf 25
> > > > SSL_connect:SSLv3 write finished A
> > > > SSL_connect:SSLv3 flush data
> > > > <<< TLS 1.0 Alert [length 0002], warning internal_error
> > > >     01 50
> > > > SSL3 alert read:warning:internal error
> > > > SSL_connect:error in SSLv3 read finished A
> > > > SSL_connect:error in SSLv3 read finished A
> > > > write:errno=104
> > > > 
> > > > 
> > > > Anyone that can help?
> > > 
> > > It would be easier if you explain exactly what you want to achieve. Also
> > > don't expect the test tools to be usable for anything else but testing
> > > Mono's SSL implementation.
> > > 
> > 
> > OK. Here is the goals for the communication:
> > 
> > My ultimate goal is to do .NET Remoting calls using https://... 
> > from my Win32 MS .NET client(s) to my Linux (Fedora 3 Core) 
> > Mono server application. 
> > (Have got my app working with http but not with https).
> > 
> > SSL should be used with both encryption and authentication.
> > (I.e. HttpWebRequest and HttpWebResponse).
> > 
> > Mutual authentication shall be done.
> > 
> > For the authentication probably an custom channel have to be used.
> 
> I don't see why this would be required (for client-side certificates) as
> the authentication is part of the protocol itself.
> 

OK. Maybe I do not need this.
I just read some articles/samples at msdn about 
.NET Remoting Authentication 
and those articles suggested a custom channel.
(http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndotnet/html/remsspi.asp and http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndotnet/html/remsec.asp
and
http://msdn.microsoft.com/msdnmag/issues/03/06/NETRemoting/default.aspx
). 
The examples was about other forms of authentication (Kerberos etc) 
and encryption so these references may not apply?.

> > (Here there seems to be interoperability problem between .NET 
> > and Mono regarding custom header naming rules, see bugg #75886.
> > .NET seems to prefer names of the form xxx_xxxx while 
> > Mono prefer names of the form xxx:xxx.
> > Hopefully I do not have to use custom names).
> > 
> > The root certificate shall be self signed.
> 
> In general all roots are self-signed (unless their public key is
> hard-coded into an application). Are you talking about the server
> certificate ? and/or the client certificates ? or a CA certificate used
> to sign other certificates ?
> 

We intend to sign both our server and client certificates 
with our own root CA.
There is already unix scripts for this using openssl, 
hence the preference for openssl.

> > The encryptions level should be relatively high.
> > Since we are not in US, maybe the exportable thing may cause problems?.
> > Preferable the key size should be 1024, but now I have gone down 
> > to 512 because of this.
> 
> IANAL but (in general) if you're not in the US then US laws don't apply
> *and* US laws are about _exporting_ cryptography not about _importing_
> it. OTOH many other country do have their own import (less frequent)
> and/or (more generally) export rules and you should comply with your own
> country laws.

OK. I do not know much about this.
I just saw some comments about this related to the PKCS#12 format 
(http://www.drh-consultancy.demon.co.uk/pkcs12faq.html)
and thought that *MAYBE* this could be a problem 
and decided to reduce this possible source of trouble.

> 
> > Binary formating shall be used.
> 
> Was there a specific reason for using https and a binary formatting ? If
> I understand correctly you'll be supplying your own server software ? so
> you _could_ use SSL without using HTTP(S).

Firewalls.

> 
> > The calls uses only primitive datatypes, byte arrays or datasets 
> > in its parameters and returns.
> > 
> > In one call a (possible large) zip file should be uploaded 
> > to the server, with as few roundtrips as possible 
> > (technique for that fixed).
> > 
> > The calls no longer have any need for callbacks or delegation.
> > 
> > Since we mostly are a linux shop the certificates is today generated 
> > with openssl.
> 
> That shouldn't be a big problem as you probably only have one (or very
> few) server certificates... 
> 
> unless you generating on linux the client certificates ???

We intend to do that.

> 
> > The server is in house and should be a Linux one.
> > We use latest version on Mono (1.1.9).
> 
> Is the server software using Mono.Security.dll ? 
> or is it using (vanilla or custom) XSP ?

Currently Mono.Security.dll.

The intention is to run the services as "Windows services" in Mono,
if that works.
I have not got into this yet, so we might have to shift.
Currently I host the remote objects in a console application.

> 
> > The clients are Win32 clients using MS .NET 
> > (v1.0 or v1.1 with SP enough to handle the certificates).
> > 
> > The clients are not connected all the time, but only during 
> > initialization and zip file transfer.
> > 
> > The installation on the client side should be as simple as possible 
> > for the clients. 
> > Hence, preferable PKCS#12 client certificates should be used 
> > and it should preferable be stored only in one file or in one store 
> > (Windows store).
> 
> You'll either have to :
> 
> (a) write your own glue (p/invokes) if you want Mono.Security.dll to
> co-exists with Windows certificate stores;
> 
> (b) completely skip Mono.Security.dll on the client side. I.e. once
> installed "correctly" the MS runtime should be able to "find" the
> private key matching the certificate you use in HttpWebRequest.

OK. I have gone for (b).

First I set the ServicePointManager.CertificatePolicy 
to an custom class to detect any certificate errors.

Then I use a DLL from Mentalis (Org.Mentalis.Security.dll) 
to extract the client certificates from the Windows store, 
and then select on the IssuerName.
Once that done, I use the Mentalis Certificate.ToX509() method 
to convert to X509Certificate(s).
(See: http://www.mentalis.org/ and 
http://www.mentalis.org/soft/projects/certificates/ ).

Last I create the HttpWebRequest, 
Add the X509Certificate(s) the requests ClientCertificates 
and set the WebResponse to the HttpWebRequest.GetResponse().
It seems to work (on the client side). 
No more "Untrusted root", since the certificates 
is in the Windows store now.

Other options for extracting the certificate(s) 
from the Windows store seems to be: 

Using CryptoAPI calls:
http://support.microsoft.com/default.aspx?scid=kb;en-us;895971

Using a CAPICOM wrapper: 
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dncapi/html/netcryptoapi.asp

Using WS-Security with the Web Service Developement Kit:
http://www.codeproject.com/csharp/cryptography.asp

> > As a first step, I would like to have the HttpWebRequest 
> > and HttpWebResponse to work toghther with HTTPS.
> 
> MS version of HttpWebRequest.ClientCertificates will "try to find"
> private keys (in it's store) associated with the certificate(s) you're
> supplying.
> 
> Note that you may have problems if the client application runs on
> another identity (e.g. services runs as SYSTEM).

OK. (Seems no problem for me).

> 
> Then, on the server side, you'll get this certificate (from
> SslServerStream) and you can validate it (with your own code) and decide
> what to do next...

Here I still have problems.

I have created a mssslserver2.exe based on your mssslserver.exe.
The difference is that mssslserver2.exe uses 
a PKCS#12-filename and a password as arguments, 
instead of a .cer-filename and .pvk-filename.
I have change the internals accordingly.

When I run this program with: 

$ mono --debug --verbose mssslserver2.exe server16-cert.p12 'password'

and call this with my client HttpWebRequest, 
I got this error message on my server: 

    error #-2146762486

meaning a CERT_E_CHAINING problem 
(also: X509ChainStatusFlags.UntrustedRoot 
and AlertDescription.UnknownCA).

I am pretty sure I have the right cert, key and CA in the .p12 file.
It seems that the root CA could not be found if it is in a PKCS#12 file.
Should I add a PKCS#12 cert file to the Mono Store (Machine store) 
(mono /.../certmgr.exe -add -c -m Trust server16-cert.p12) 
or add only the CA to the trust store
(mono /.../certmgr.exe -add -c -m Trust cacert16t.cer)
or anything else to get rid of this error message?.

On the server I get an exception during the client call 
(in: reader.ReadLine ()) below: 
>>>
...
using (SslServerStream s = new SslServerStream (ns, Certificate, true, 
false)) {
... 
StreamReader reader = new StreamReader (s);
...
string line;
// Read request header 
do {
        line = reader.ReadLine ();
...
<<<

The exception goes like this:
>>>
EXCEPTION handling: TlsException
EXCEPTION handling: TlsException
EXCEPTION handling: IOException
EXCEPTION handling: IOException
---------------------------------------------------------
System.IO.IOException: The authentication or decryption has failed. --->
Mono.Security.Protocol.Tls.TlsException: Handshake Failure.
in <0x00134> 
Mono.Security.Protocol.Tls.Handshake.Server.TlsClientCertificateVerify:ProcessAsSs3  ()
in <0x00057> 
Mono.Security.Protocol.Tls.Handshake.HandshakeMessage:Process ()
in (wrapper remoting-invoke-with-check) 
Mono.Security.Protocol.Tls.Handshake.HandshakeMessage:Process ()
in <0x00084> 
Mono.Security.Protocol.Tls.ServerRecordProtocol:ProcessHandshakeMessage 
(Mono.Security.Protocol.Tls.TlsStream handMsg)
in <0x00239> 
Mono.Security.Protocol.Tls.RecordProtocol:InternalReceiveRecordCallBack 
(IAsyncResult asyncResult)--- End of inner exception stack trace ---

in <0x000d4> 
Mono.Security.Protocol.Tls.SslStreamBase:AsyncHandshakeCallback
(IAsyncResult asyncResult)
<<<

I guess that this has to do with the CERT_E_CHAINING problem 
mentioned above. Possible?. Right?

> 
> > As a second step, I will try to do this using .NET Remoting.
> 
> I don't know enough about remoting (like it's support for
> HttpWeb[Request|Response] to guide you but I expect it will be similar.
> 
> > > > May my problems be because i use openssl?
> > > 
> > > It's likely for the PVK-related problems.
> > 
> > I will change to PKCS#12 format.
> > 
> > > 
> > > > Anyone that have done a working HTTPS call from a MS .NET client 
> > > > to a Linux Mono server before?.
> > > 
> > > a *big* lot ;-)
> > > but far fewer have used client certificates
> > > 
> > 
> > OK. Those that have done this does not seem to have documented 
> > it that well on the internet though :-).
> 
> Sadly the ratio of people sharing their errors/problems is much higher
> than the ratio of people sharing their solutions ;-)
> 
> I don't generally blog about MS specific stuff (i.e. the private key
> association to a certificate in their store) but I'll try to make this
> work and "enrich" the Internet ;-)

Much tanks for that.
Now a solution seems reachable.

I have tries to fill this posting with some more references than normal,
to gather this information in one place.

> 
> > > > FYI, On the client i run Microsoft Windows 2000, SP4 
> > > > (emulated with WMware workstation).
> 
> This shouldn't be a problem.

Regards 

Yngve Zackrisson.