AS2MDNTroubleshootingEDI

AS2 insufficient-message-security Error: 5 Causes and How to Fix Each One

7 min readBy Marwan Nakhaleh

Your AS2 message went out. An MDN came back. But instead of a clean receipt, the disposition line reads automatic-action/MDN-sent-automatically; processed/error: insufficient-message-security. Your partner's server received the message, looked at how it was secured, and rejected it.

The good news: this is one of the more honest AS2 errors. It means exactly one thing. The security layers on your outbound message do not meet your partner's policy. No guessing about firewalls or DNS. The message arrived. It just was not protected the way they demand.

The bad news: the error never tells you which layer fell short. Signing? Encryption? The algorithm? Here are the five causes, how to confirm each one, and the fix.

First, know where to look

The insufficient-message-security string is an MDN disposition modifier defined in RFC 4130. Find it in the Disposition field of the machine readable part of the MDN your partner returned. Your AS2 server logs it; in OpenAS2 it shows up in the MDN text stored alongside the sent message, in mendelson it appears in the transaction detail view.

If you got no MDN at all, this is a different problem. Work through AS2 MDN not received instead. And if the MDN arrived but your side failed to verify it, that is covered in MDN verification failed.

Cause 1: The message is not signed and your partner requires signing

How to confirm: Check the Content-Type header of your outbound message in your AS2 server's raw send log. A signed message shows multipart/signed (or application/pkcs7-mime with smime-type=signed-data when encryption wraps it). If you see only application/edi-x12, application/edifact, or a bare application/pkcs7-mime; smime-type=enveloped-data with no signature inside, you sent unsigned.

The fix: Turn on signing for that partnership. In OpenAS2, set the sign attribute in the partnership definition (use sha256). In mendelson, enable Sign in the partner's security settings. Then confirm the signing certificate you use is the one your partner has on file.

Cause 2: The message is not encrypted and your partner requires encryption

How to confirm: Same log, same header. An encrypted message carries Content-Type: application/pkcs7-mime; smime-type=enveloped-data. If your payload's real content type is visible in the outbound request, it went out in the clear.

The fix: Enable encryption with your partner's public certificate. In OpenAS2 that is the encrypt attribute on the partnership (use aes256 unless the partner specifies otherwise). Double check you encrypt with the partner's certificate, not your own. That mix up produces a different failure on their side, covered in AS2 decryption failed.

Cause 3: You enabled one layer but the partner requires both

Plenty of trading partners, especially in pharma and large retail, require sign and encrypt together. Signed but unencrypted, or encrypted but unsigned, still trips the policy.

How to confirm: Pull the partner's connectivity spec sheet and read the security section literally. Then compare against your outbound Content-Type. Encrypted plus signed looks like enveloped-data wrapping a multipart/signed body, which you can only see after decryption, so trust your partnership config here: both sign and encrypt set.

The fix: Set both attributes. Send a test. Read the new MDN disposition.

Cause 4: Your algorithm is below the partner's floor

You are signing and encrypting, but with SHA-1 and 3DES. Many gateways now reject anything below SHA-256 and AES-128, and some return insufficient-message-security rather than a more specific error.

How to confirm: Check the exact algorithm values in your partnership config. In OpenAS2, sign="sha1" or encrypt="3des" are the usual suspects, often left over from a config copied years ago. Ask the partner for their minimums in writing.

The fix: Move to sha256 for signing and aes256 for encryption. Nearly every AS2 implementation from the last decade supports both. This is the single most common resolution for this error.

Cause 5: The partner tightened their policy and told nobody

The connection worked for three years. Nothing changed on your side. Now every message bounces with insufficient-message-security. Security teams roll out new minimum standards on gateway upgrades, and notification emails go to a distribution list nobody reads.

How to confirm: Establish the exact timestamp of the first failure and ask your partner whether a gateway upgrade or policy change deployed that day. If your config diff shows nothing on your side, the policy moved under you.

The fix: Get the partner's current requirements in writing, update your partnership settings to match, and send a test message before resuming production traffic. Treat any partner platform migration announcement as a trigger to retest, not just a heads up.

Stop guessing which cause it is

Every cause above hides behind the same nine word error. Confirming them one at a time means log diving on your side, emails to the partner, and a day of back and forth.

There is a faster way to check your half of the equation. Run the free AS2 connection test. It sends a real AS2 message against your endpoint and grades signing, encryption, algorithms, certificates, TLS, and MDN handling individually, so you can see exactly which security layer is configured and which is missing before your partner's gateway tells you again.

For the full methodology, the guide on how to test an AS2 connection walks through every check. 60 seconds. One answer. No more reading tea leaves in a disposition field.