Signing Soap message and its attachments by WSS4J to further use (for example) by SoapUI

In this post it is shown how to make SOAP message signing with use of the following Java library WSS4J version 2.1.x.

The topic is not so widely described and may be helpful for people who try to implement it but find issues when something has to be done more custom way. In our case performing signing in Java was a necessity due to limitation of SoapUI tool that particularly didn't have attachments signing. As a workaround, from high-level perspective we did it the following way:

1. Java project is being used for message signing and action is done through main function having parameters. Depending on number of parameters specific sub-action of signing is being done because we have to support different kinds of soap messages (PUSH, PULL, RECEIPT, RECEIPT_ERROR). PUSH is special message type because requires signing of not only the payload but also attachments. This was main the reason of creating the signing procedure outside of SoapUI.
The functionality is developed in Java project and by maven-shade-plugin exported to executable Java JAR file. This way it is standalone JAR that has all necessary dependencies (other dependent JARs).

2. SoapUI executes the jar by "java - jar "parameter1" "parameter2" "parameterX" like for example:

java -jar messageSigner.jar push "keystoreAlias|C:/Users/userName/svn_repository/certificates/keystore1.JKS|password" "C:/Users/userName/svn_repository/TMP/requestToSign.xml" "AttachmentAlias=C:/Users/userName/svn_repository/resources/TMP/attachment1.xml.gz|AttachmentAlias2=C:/Users/userName/svn_repository/resources/TMP/attachment2.xml.gz"

The parameters are parsed so that "|" (pipe) is special character to make a split, also the first parameter determines how to parse further parameters and what to expect. Note that this approach allows for any number of attachments. The only matter is for SoapUI to prepare appropriate parameters (based on actual state in SoapUI step, request attachments in the test step) and the Java to interpret the args[] in proper way. Here a question arises: why didn't we put the JAR to bin ext and loaded it into SoapUI classpath? The answer is the following: some WSS4j2.0 dependent library (xml-sec-2.0.5.jar) was in conflict with the library used by SoapUI (xml-sec-1.4.5.jar). Replacing SoapUI's lib with the newer xmlsec.jar resulted in that loading of our messageSigner.jar ws posible but SoapUI basic functionality stopped working. If we tackled that problem then it would be easier to work with external jar and have access to objects. Also it would run faster and there would be initially no synchronization issues on the SoapUI. side.But pressure of time and necessity to have working solution forced us to make it via "java -jar" command and running 'main' function with parameters. In the parameters please also note a path to unsigned request, because the JAR needs unsigned XML as an input. Preparing the arguments for the JAR and handling the output will be described in details in the separate article.
3. Java returns signed request as an output. This output is being placed in the request form in SoapUI step (made by Groovy). The original unsigned request is stored under variable.
4. After the request is executed the very first groovy assertion restores the original unsigned soap request. It is made that way to have it executed even if there is an error. We need to have the same test script after execution. Of course using SoapUI is not necessary and we can sign, send, assert requests through Java directly. But we had already plenty of test cases in SoapUI, so took (not optimal looking from perspective of time) decision to still utilize SoapUI tool. The recommendation is to use just Java and refrain from using tools if the team consists of people who know programming.

Below is the implementation of the Java project.
1. List of dependencies in the POM.xml:

    <dependencies>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>2.1</version>
            <type>jar</type>
        </dependency>
        <dependency>
            <groupId>org.apache.wss4j</groupId>
            <type>pom</type>
            <artifactId>wss4j</artifactId>
            <version>2.1.3</version>
        </dependency>
        <dependency>
            <groupId>org.apache.wss4j</groupId>
            <artifactId>wss4j-ws-security-common</artifactId>
            <version>2.1.3</version>
            <type>jar</type>
        </dependency>
        <dependency>
            <groupId>org.apache.wss4j</groupId>
            <artifactId>wss4j-ws-security-dom</artifactId>
            <version>2.1.3</version>
            <type>jar</type>
        </dependency>
        <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
            <version>2.4</version>
            <type>jar</type>
        </dependency>
    </dependencies>

2. Maven shade plugin configuration in POM.xml:

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>2.4.1</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
            <configuration>
              <transformers>
                <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                  <manifestEntries>
                    <Main-Class>com.messageSigner.security.WSSSignPushMessage</Main-Class>
                    <Build-Number>123</Build-Number>
                  </manifestEntries>
                </transformer>
              </transformers>
              <filters>
        <filter>
            <artifact>*:*</artifact>
            <excludes>
                <exclude>META-INF/*.SF</exclude>
                <exclude>META-INF/*.DSA</exclude>
                <exclude>META-INF/*.RSA</exclude>
            </excludes>
        </filter>
    </filters>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

3. Helper method for reading a file (for sure there are different libraries to read from a file, this is one of possibilities):

public static String readFile(String path) throws IOException {
 File file = new File(path);
 StringBuilder fileContents = new StringBuilder((int)file.length());
 Scanner scanner = new Scanner(file);
 String lineSeparator = System.getProperty("line.separator");
 try {
  while(scanner.hasNextLine()) {        
   fileContents.append(scanner.nextLine() + lineSeparator);
  }
  return fileContents.toString();
 } finally {
  scanner.close();
 }
}

4. Content of a AttachmentCallbackHandler.java (found in Apache documentation):

import javax.security.auth.callback.Callback;
import javax.security.auth.callback.CallbackHandler;
import javax.security.auth.callback.UnsupportedCallbackException;

import org.apache.wss4j.common.ext.Attachment;
import org.apache.wss4j.common.ext.AttachmentRequestCallback;
import org.apache.wss4j.common.ext.AttachmentResultCallback;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class AttachmentCallbackHandler implements CallbackHandler {

    private final List<Attachment> originalRequestAttachments;
    private Map<String, Attachment> attachmentMap = new HashMap<>();
    private List<Attachment> responseAttachments = new ArrayList<>();

    public AttachmentCallbackHandler() {
        originalRequestAttachments = Collections.emptyList();
    }

    public AttachmentCallbackHandler(List<Attachment> attachments) {
        originalRequestAttachments = attachments;
        if (attachments != null) {
            for (Attachment attachment : attachments) {
                attachmentMap.put(attachment.getId(), attachment);
            }
        }
    }

    public void handle(Callback[] callbacks)
            throws IOException, UnsupportedCallbackException {
        for (int i = 0; i < callbacks.length; i++) {
            if (callbacks[i] instanceof AttachmentRequestCallback) {
                AttachmentRequestCallback attachmentRequestCallback =
                        (AttachmentRequestCallback) callbacks[i];

                List<Attachment> attachments =
                        getAttachmentsToAdd(attachmentRequestCallback.getAttachmentId());
                if (attachments.isEmpty()) {
                    throw new RuntimeException("wrong attachment requested");
                }

                attachmentRequestCallback.setAttachments(attachments);
            } else if (callbacks[i] instanceof AttachmentResultCallback) {
                AttachmentResultCallback attachmentResultCallback =
                        (AttachmentResultCallback) callbacks[i];
                responseAttachments.add(attachmentResultCallback.getAttachment());
                attachmentMap.put(attachmentResultCallback.getAttachment().getId(),
                        attachmentResultCallback.getAttachment());
            } else {
                throw new UnsupportedCallbackException(callbacks[i], "Unrecognized Callback");
            }
        }
    }

    public List<Attachment> getResponseAttachments() {
        return responseAttachments;
    }

    // Try to match the Attachment Id. Otherwise, add all Attachments.
    private List<Attachment> getAttachmentsToAdd(String id) {
        List<Attachment> attachments = new ArrayList<>();
        if (attachmentMap.containsKey(id)) {
            attachments.add(attachmentMap.get(id));
        } else {
            if (originalRequestAttachments != null) {
                attachments.addAll(originalRequestAttachments);
            }
        }

        return attachments;
    }
}

5. Content of the WSSSigner.java

import com.company.commons.FileSystemMethods;
import org.apache.commons.io.IOUtils;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.wss4j.common.WSEncryptionPart;
import org.apache.wss4j.common.crypto.Crypto;
import org.apache.wss4j.common.crypto.CryptoFactory;
import org.apache.wss4j.common.ext.Attachment;
import org.apache.wss4j.common.util.AttachmentUtils;
import org.apache.wss4j.dom.WSConstants;
import org.apache.wss4j.dom.message.WSSecHeader;
import org.apache.wss4j.dom.message.WSSecSignature;
import org.w3c.dom.Document;
import javax.activation.DataHandler;
import javax.activation.FileDataSource;
import javax.xml.soap.*;
import javax.xml.transform.stream.StreamSource;
import java.io.*;
import java.security.KeyStore;
import java.security.Security;
import java.util.*;
import java.util.regex.Pattern;

public class WSSSigner {

    private static final Logger logger = LogManager.getLogger();

    private static String keystoreFile;
    private static String keystorePass;
    private static String keystoreAlias;
    private static String truststoreFile;
    private static String truststorePass;

    //create Properties for crypto
    private Properties propertiesCrypto = null;
    private static Crypto crypto;
    private static String signedSoapMessage;
    private static String requestType;

    public WSSSigner(HashMap mapKeystores) throws Exception{

        //this is required to prevent exception coming from BouncyCastle: "Caused by: java.lang.SecurityException: JCE cannot authenticate the provider BC ..."
        Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider());

        keystoreFile = mapKeystores.get("keystoreFile");
        keystorePass = mapKeystores.get("keystorePass");
        keystoreAlias = mapKeystores.get("alias");
        truststoreFile = mapKeystores.get("truststoreFile");
        truststorePass = mapKeystores.get("truststorePass");

        System.setProperty("javax.net.ssl.keyStore", keystoreFile);
        System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
        //System.setProperty("javax.net.ssl.trustStore", truststoreFile);
        //System.setProperty("javax.net.ssl.trustStorePassword", truststorePass);

        this.propertiesCrypto = new Properties();
        this.propertiesCrypto.setProperty("org.apache.wss4j.crypto.provider", "org.apache.wss4j.common.crypto.Merlin");

        //quite naive but working method to recognize certificate type (only pfx and jks supported)
        String certificateExtension = keystoreFile.substring(keystoreFile.length()-3);
        if(certificateExtension.equalsIgnoreCase("pfx")) {
            this.propertiesCrypto.setProperty("org.apache.wss4j.crypto.merlin.keystore.type", "pkcs12");
        } else if (certificateExtension.equalsIgnoreCase("jks")) {
            this.propertiesCrypto.setProperty("org.apache.wss4j.crypto.merlin.keystore.type", "jks");
        } else {
            throw new Exception("certificate extension can be only jks or pfx while it was '" + certificateExtension + "'");
        }

        //=================================Extracting alias from keystore programatically========
        //certificate alias is provided in arguments but in case when there is only 1 then it is being used
        //it's not optimal but made like that to provide backward compatibility with the code in Groovy for SoapUI
        String alias = null;

        File file = new File(mapKeystores.get("keystoreFile"));
        FileInputStream is = new FileInputStream(file);
        KeyStore keystore = KeyStore.getInstance(this.propertiesCrypto.getProperty("org.apache.wss4j.crypto.merlin.keystore.type"));
        keystore.load(is, mapKeystores.get("keystorePass").toCharArray());

        int aliasCounter = 0;
        Enumeration enumeration = keystore.aliases();
        while(enumeration.hasMoreElements()) {
            alias = (String)enumeration.nextElement();
            aliasCounter++;
            logger.error("alias" + aliasCounter + " name: " + alias);
            //Certificate certificate = keystore.getCertificate(alias); //not useful here but for potential debugging
            //System.out.println(certificate.toString());
        }

        if (aliasCounter > 1) {
            logger.error(aliasCounter + " aliases exist, while expecting just one, so using the alias from external parameter: " + keystoreAlias);
            this.propertiesCrypto.setProperty("org.apache.wss4j.crypto.merlin.keystore.alias", keystoreAlias); //using external alias value, not trying to guess alias name
        } else {
            logger.error(aliasCounter + " alias exist, selecting the one resolved programatically.");
            mapKeystores.remove("alias");
            mapKeystores.put("alias", alias);
            keystoreAlias = mapKeystores.get("alias");
            this.propertiesCrypto.setProperty("org.apache.wss4j.crypto.merlin.keystore.alias", alias); //using alias just from certificate
        }
        //=======================================================================================

        this.propertiesCrypto.setProperty("org.apache.wss4j.crypto.merlin.keystore.password", keystorePass);
        this.propertiesCrypto.setProperty("org.apache.wss4j.crypto.merlin.keystore.file", keystoreFile);
        crypto = CryptoFactory.getInstance(propertiesCrypto);
    }
    
    //example main where input args are being extracted
    //here 4 types of requests wehere there: pull, receipt, error (receipt but with error status), push
    //push action is responsible for sending message with attachment to the node, that's why it is treated separately
    public static void main(String args[]) throws Exception {

        HashMap attachments = new HashMap<>();
        
        if(args.length>0)
            requestType = args[0];
        
        if (args.length == 3) {
            if (requestType.equalsIgnoreCase("pull") || requestType.startsWith("receipt") || requestType.equalsIgnoreCase("error")) {
                //correct parameters
            } else {
                throw new Exception("Method main needs first argument as: 'pull' 'receipt' or 'error' in case of 3 input parameters");
            }
        } else if (args.length == 4) {
            if (requestType.equalsIgnoreCase("push")) {
                String rawAttachInfo = args[3];
                String[] attachmentsRaw = rawAttachInfo.split(Pattern.quote("|"));

            for (String attachmentRaw : attachmentsRaw) {
                String[] attachmentRawCoupled = attachmentRaw.split(Pattern.quote("="));
                attachments.put(attachmentRawCoupled[0], attachmentRawCoupled[1]);
            }
            } else {
                throw new Exception("Method main needs first argument as: 'push' in case of 4 input parameters");
            }
        } else {
            throw new Exception("Method main needs 3 or 4 input parameters while it got: " + args.length);
        }

        String[] aliasKeyStorTruststoreInfo = args[1].split(Pattern.quote("|"));

        HashMap mapKeystores = new HashMap<>();
        mapKeystores.put("alias", aliasKeyStorTruststoreInfo[0]);
        mapKeystores.put("keystoreFile", aliasKeyStorTruststoreInfo[1]);
        mapKeystores.put("keystorePass", aliasKeyStorTruststoreInfo[2]);
        WSSSigner myMsgToSign = new WSSSigner(mapKeystores);

        String pathToUnsignedReqOrPullResponse = args[2];
        myMsgToSign.getSignedSoapMessageAsString(attachments, pathToUnsignedReqOrPullResponse);
        System.out.print(signedSoapMessage);  //this output is the clue of the application because SoapUI is grabbing it to its 'Request' field during Send action
    }
    
    //for troubleshooting
    public String helloWorld() throws Exception {
        //String sb = readFileFromClassLocation("/template_receipt.xml");
        //System.out.println("##################" + sb);
        return "Hello world from java jar file";
    }

    //used when some files are inside the JAR file
    // may be used for reading static templates that are not dependent on dynamic values
    public String readFileFromClassLocation(String path) throws Exception {
        String HoldsText;
        InputStream is = getClass().getResourceAsStream(path);
        InputStreamReader fr = new InputStreamReader(is);
        BufferedReader br = new BufferedReader(fr);

        StringBuilder sb = new StringBuilder();
        while((HoldsText = br.readLine())!= null){
            sb.append(HoldsText)
                    .append("\n");
        }
        return sb.toString();
    }

    public String getSignedSoapMessageAsString(HashMap attachments, String pathToUnsignedReqOrPullResponse) throws Exception {
        createSoapMessage(attachments, pathToUnsignedReqOrPullResponse);
        return signedSoapMessage;
    }
    
    public SOAPMessage createSoapMessage(HashMap attachments, String pathToUnsignedReqOrPullResponse) throws Exception {

        // Create message
        MessageFactory mf = MessageFactory.newInstance(SOAPConstants.SOAP_1_2_PROTOCOL);
        SOAPMessage msg = mf.createMessage();
        String fileContent = FileSystemMethods.readFile(pathToUnsignedReqOrPullResponse);

        //part for 'positive receipt' and 'error receipt'
        if (WSSSigner.requestType.equalsIgnoreCase("receipt")) {
            String template = readFileFromClassLocation("/template_receipt.xml");
            fileContent = XmlTransformation.preparePositiveReceipt(template, fileContent, "receipt");

        } else if (WSSSigner.requestType.equalsIgnoreCase("error")) {
            String template = readFileFromClassLocation("/template_error.xml");
            fileContent = XmlTransformation.prepareErrorReceipt(template, fileContent);
        }

        // Object for message parts
        SOAPPart soapPart = msg.getSOAPPart();
        InputStreamReader isr = new InputStreamReader(IOUtils.toInputStream(fileContent));
        StreamSource prepMsg = new StreamSource(isr);
        soapPart.setContent(prepMsg);
        
        for (Map.Entry entry : attachments.entrySet()) {
            String contentID = entry.getKey();
            String filePath = entry.getValue();
            
            //adding attachment part
            File f = new File(filePath);
            DataHandler dh = new DataHandler(new FileDataSource(f));
            AttachmentPart objAttachment = msg.createAttachmentPart(dh);
            objAttachment.setContentId("<" + contentID + ">");
            objAttachment.setContentType("application/gzip");
            objAttachment.setMimeHeader("Content-Transfer-Encoding", "binary");
            objAttachment.setMimeHeader("Content-Disposition", "attachment");

            msg.addAttachmentPart(objAttachment);
        }

        //saving message
        if(!requestType.equalsIgnoreCase("pull"))
            msg.saveChanges();

        logger.debug("Request SOAP Message:");
        msg.getSOAPPart().setTextContent(signSOAPEnvelope(msg));
        msg.saveChanges();
        return msg;
    }

    //most important method that is subject of this article
    //here the signing takes place
    public String signSOAPEnvelope(SOAPMessage unsignedMessage) throws Exception {
        
        SOAPEnvelope unsignedEnvelope = unsignedMessage.getSOAPPart().getEnvelope();
        Document doc = unsignedEnvelope.getOwnerDocument();

        WSSecSignature signer = new WSSecSignature();
        WSSecHeader secHeader = new WSSecHeader(doc);
        secHeader.setMustUnderstand(true);
        
        signer.setUserInfo(keystoreAlias, keystorePass);

        //parameters below depend on the algorithms being used by the server, so treat it as example
        //otherwise use different enums/values
        signer.setSignatureAlgorithm("http://www.w3.org/2001/04/xmldsig-more#rsa-sha256");
        signer.setSigCanonicalization(WSConstants.C14N_EXCL_OMIT_COMMENTS);
        signer.setDigestAlgo(WSConstants.SHA256);
        signer.setKeyIdentifierType(WSConstants.BST_DIRECT_REFERENCE);
        signer.setAddInclusivePrefixes(false);

        signer.appendBSTElementToHeader(secHeader); //appending Binary Security Token

        signer.setUseSingleCertificate(true);
        
        //signing specific parts, likewise in SoapUI
        signer.getParts().add(new WSEncryptionPart("Body", "http://www.w3.org/2003/05/soap-envelope","Element"));
        signer.getParts().add(new WSEncryptionPart("Messaging", "http://docs.oasis-open.org/ebxml-msg/ebms/v3.0/ns/core/200704/","Element")); 
                
        //create list of attachments to be signed, functionality that was lacking in SoapUI and all the fuzz was about
        Iterator iterator = unsignedMessage.getAttachments();
        List listAttachments = new ArrayList();
        boolean attachFiles = false;
        while(iterator.hasNext()) {
            attachFiles = true;
            AttachmentPart atttachmentPart = (AttachmentPart)iterator.next();
            Attachment attachment = new Attachment();
            attachment.addHeaders(getHeaders());
            attachment.setId(atttachmentPart.getContentId().replaceAll("<|>", ""));
            attachment.setSourceStream(atttachmentPart.getDataHandler().getDataSource().getInputStream());
            listAttachments.add(attachment);
        }
        
        if(attachFiles) {
            signer.getParts().add(new WSEncryptionPart("cid:Attachments", "Content"));
            AttachmentCallbackHandler attachmentCallbackHandler = new AttachmentCallbackHandler(listAttachments);
            signer.setAttachmentCallbackHandler(attachmentCallbackHandler);
        }
        
        secHeader.insertSecurityHeader();
        signer.prepare(doc, crypto, secHeader);

        Document signedDoc = signer.build(doc, crypto, secHeader);

        signedSoapMessage = org.apache.wss4j.common.util.XMLUtils.PrettyDocumentToString(signedDoc);
        //logger.info("Signed message: \n\r\n\r" + signedSoapMessage + "\n\r");
        
        return signedSoapMessage;
    }

    public static Map getHeaders() {
        Map headers = new HashMap<>();
        headers.put(AttachmentUtils.MIME_HEADER_CONTENT_DISPOSITION, "attachment");
        headers.put(AttachmentUtils.MIME_HEADER_CONTENT_TYPE, "application/gzip");
        headers.put("Content-Transfer-Encoding", "binary");
        return headers;
    }
}