MD4 hashlib support in Python 3.8

May 2024 ยท 3 minute read

I am trying to implement a soap client for a server that uses NTLM authentication. The libraries that I use (requests-ntlm2 which relies on ntlm-auth) implement the MD4 algorithm that lies in the core of the NTLM protocol via the standard library's hashlib.

Although hashlib seems to support MD4:

>>> import hashlib >>> hashlib.algorithms_available {'md5-sha1', 'md4', 'shake_128', 'md5', 'blake2s', 'sha3_512', 'ripemd160', 'sha512', 'mdc2', 'blake2b', 'sha3_256', 'sha3_224', 'sha512_224', 'sha1', 'sha384', 'sha256', 'sha224', 'whirlpool', 'sha512_256', 'sha3_384', 'shake_256', 'sm3'} >>> 

and so does the openssl library in my system:

(victory) C:\code\python\services>openssl help: [...] Message Digest commands (see the `dgst' command for more details) blake2b512 blake2s256 md4 md5 mdc2 rmd160 sha1 sha224 sha256 sha3-224 sha3-256 sha3-384 sha3-512 sha384 sha512 sha512-224 sha512-256 shake128 shake256 sm3 [...] 

when the authentication tries to run python produces an ValueError: unsupported hash type md4 error. Here is the relevant part of the traceback:

C:\ProgramData\Miniconda3\envs\victory\lib\site-packages\ntlm_auth\compute_hash.py in _ntowfv1(password) 165 return nt_hash 166 --> 167 digest = hashlib.new('md4', password.encode('utf-16-le')).digest() 168 169 return digest C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py in __hash_new(name, data, **kwargs) 161 # This allows for SHA224/256 and SHA384/512 support even though 162 # the OpenSSL library prior to 0.9.8 doesn't provide them. --> 163 return __get_builtin_constructor(name)(data) 164 165 C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py in __get_builtin_constructor(name) 118 return constructor 119 --> 120 raise ValueError('unsupported hash type ' + name) 121 122 ValueError: unsupported hash type md4 

Even when I try to merely call the MD4 from hashlib, I get the same result:

>>> import hashlib >>> hashlib.new('md4') Traceback (most recent call last): File "C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py", line 157, in __hash_new return _hashlib.new(name, data) ValueError: [digital envelope routines] initialization error During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py", line 163, in __hash_new return __get_builtin_constructor(name)(data) File "C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py", line 120, in __get_builtin_constructor raise ValueError('unsupported hash type ' + name) ValueError: unsupported hash type md4 

Any insights about what's going on and/or any help would be immensely appreciated.

7

5 Answers

For ubuntu (jammy/focal).

Add this to your /etc/ssl/openssl.cnf to 're-enable' md4 to hashlib

[provider_sect] default = default_sect legacy = legacy_sect [default_sect] activate = 1 [legacy_sect] activate = 1 

Solution from this https://bugs.launchpad.net/ubuntu/+source/python3.10/+bug/1971580/comments/3

Depending on your version or distribuition the path to the openssl.cnf file can be /usr/lib/openssl.cnf or other.

3

Update as of May 2023

This has been addressed in requests-ntlm>=1.2.0 which now uses SPNEGO: https://github.com/requests/requests-ntlm/pull/126

OpenSSL configuration changes should no longer be required with this version.

Tip: Since this is one of the top results for "unsupported hash type md4" (an error I encountered when using Ansible to target Windows on a modern Linux distro) simply updating this library in your current Ansible install appears to be a solution:

python3 -m pip install -U requests-ntlm 

Well, it seems that there was something corrupted in my conda environment. I created a new identical one, and it's been working ever since without having to change anything else.

3

In my case, I was running a docker image python:3.7 and didn't specify the sub-version. The fix mentioned by @imbr did work. I found the openssl.cnf file location by running openssl version -d

But also, on my dev station, it was using python:3.7.15 and on the live server was python:3.7.17 From the release notes, you can see that openssl got an update on 17, https://www.python.org/downloads/release/python-3717/

hashlib.new('md4', "test".encode()).hexdigest() 
1

ncG1vNJzZmirpJawrLvVnqmfpJ%2Bse6S7zGiorp2jqbawutJobXJxY22CeHyOppttZZiWwKm4yJtkrK2gpbyzwIyipWaoqam1sLqMbGRx

ncG1vNJzZmikkpi%2Bb7%2FFqGpnnJmctrWty6ianpmeqL2ir8SsZZynnWS6pYCMoZisoJyer26%2F1KmnqKqkYravec%2Byq6GnnmKAboSNoaumpA%3D%3D