[Mono-list] TripleDES encryption problem

[Sébastien Pouliot]

Carlos,

> Huuummmm  i'm encrypting the fragment + mac +
> padding length, this gives 37 bytes of data and 3 bytes more for
> padding, if i'm not wrong the correct padding is

Well this isn't the structure described in rfc2246. You may not be
encrypting the right structure!
But I'm unsure why you are encrypting this. What are you using as the TLS
specification ?

> I have checked this on windows and seems to be correct.

That's still a mystery why this works under Windows !
But there's something you can try:
- Rename Mono's TripleDESCryptoServiceProvider class to something else (like
Mono3DES).
- Include this new into your projet
- Duplicate all crypto code using TripleDESCryptoServiceProvider;
- Compare all output buffers between the 2 implementations
somewhere there must be a difference between the outputs...

> And a final note from the Microsoft .NET Framework help:
>
> PaddingMode.PKCS7
>
> The PKCS #7 padding string consists of a sequence of bytes, each of
> which is equal to the total number of padding bytes added. For example,
> if 24 bits (3 bytes) of padding need to be added, the padding string is
> "03 03 03".

Yeah this is all true, PKCS#7 pads using the number of unused bytes as the
padding value. You can see this in your previous sample when you had 7 times
0x07 decoded by OpenSSL - so Mono got the PKCS#7 padding right.

But this ISN'T the behaviour described in rfc2246 in section 6.2.3.2. If you
check the included example (page 20) you'll see that the last 8 bytes are xx
06 06 06 06 06 06 06 (7 times 0x06) which ISN'T a standard PKCS#7 padding
but consistent with the pseudo-code GenericBlockCipher structure.

I hope this helps
Good luck

Sebastien Pouliot
Security Architect, Motus Technologies, http://www.motus.com/
work: spouliot@motus.com
home: spouliot@videotron.ca