Cheap credit cards

Perhaps the most enlightening part of the entire talk was that its possible to buy valid credit card numbers on the web for as little as £1. Cards with the supposedly secure CCV number can be had for less that £5, while the card with the pin number can be had for £10. That’s scary! With prices so low, its possible to see the potential volume of fraudulent transactions your web site may be subject to. Its also easy to understand why around 50% of shopping cards are abandoned at the payment stage. People simply don’t feel secure entering their details online.

Collecting payments

Still thinking of the DIY solution to collecting your own payments online? At this stage I must admit that I was in two minds, sure there’s the potential for fraud, but it can’t be too hard to detect. And anyway the banks back credit cards against fraud, so you’re covered there (although this assumption was later proved to be incorrect as you are in a cardholder not present scenario). Surely collecting some card details and transmitting a payment can’t be too difficult, banks have been doing this for some time, they must have a standardized interface?

The talk moved on to cover the pitfalls of trying to implement a DIY solution to collecting payments, after which I was left with the feeling that I certainly didn’t want to try to do this myself.

Payment acceptance

Obvious really, you need to accept a payment for the product/service you are trying to sell. However that payment needs to go somewhere, and will probably go into a merchant bank account whereupon the bank will charge you around 2.5% of the value. Ouch. What’s more you will likely need to have accounts for each currency you want to collect payments in, or suffer more charges on currency conversion.

But what about a standardized interface for banks to collect payments? Apparently not, each payment vendor may well be using their own unique proprietary interface. This in itself is a bit of a headache but could be managed with the relevant abstractions in the code. Banks require that your interactions with these interfaces be certified (and re-certified on a regular basis), which makes sense. They want you to minimize the risk of the payments failing, or you sending them rubbish. However this certification means testing, which takes time to create, run and verify, and with each bank potentially having different processes for payments, suddenly that abstraction layer is looking a little more complex.

Assuming that the interfaces can be created and that they pass the certification, there’s still a cost in their day to day usage. They must be monitored to ensure that they continue to function as expected, and they may need to be re-certified several times a year against new versions of the banking interfaces.

Order verification

Before shipping the order, the system must be satisfied that the customer is who they claim to be, and that their payment details match. In essence details such as the shipping address matching the billing address, or one of the other known addresses for that customer. That multiple different account holders haven’t been seen at the same address, or if they have that there’s a reasonable explanation for them (e.g. multiple members of the same family, shared accommodation, etc).

Orders that were verified and shipped, but later challenged through the banks may need to be contested. Resources will need to be allocated to deal with researching the transaction and the customer, and to liaise with the bank. Apparently the banks can reverse the transaction, taking the payment back out of the merchant’s account, without prior notification.

Its not possible to determine the validity of all orders through an automated system, some of them will ultimately be passed into a manual process. Cybersource estimates that this is as much as 30% of all orders, which in a successful company could equate to a large amount of time needing to be spend manually reviewing these orders. Even in a small company, its unlikely that they would be able to support people dedicated to reviewing orders. Manual review takes time as details such as addresses, phone numbers, email, IP addresses an order history all need to be taken into account when determining a particular order’s validity.

Processing Management

Actually collecting the payment from the customer without accruing additional bank charges provides more challenges. Banks may charge extra for any deviation in the standard happy path process. So if we leave it too long between getting an AUTH code back on the account and actually collecting the payment we may have to pay more.

There may be issues that cause us to have to do additional processing on our side, work beyond our normal happy path collection process. The customer’s account many not actually have the money in the account when we go to collect it, their card details may have expired, etc. These issues can become a bigger problem if we have already sent the product out.

Reconciliation

To balance the books we have to reconcile the bank’s records of transactions with our own records. Determining what products were sent that weren’t paid for, or worse what products that were paid for that weren’t sent (seriously unhappy customers there). Reversed transactions need to be recorded and our own fraud database updated so that we can get smarter about detecting them in the future. It would be nice to assume that we could automate this reconciliation, but we face the same issue of having potentially different interfaces for each bank that we deal with.

Security

Perhaps the most problematic and headache inducing area of all is that of payment security. Having payment information flowing through our system leaves us vulnerable to hackers. Its no longer the script kiddies who we have to fear, but organized gangs with some really smart people paid to hack online retailers. Could we react quick enough if a vulnerability is found in a product that we use? How long would we have to be exposed for before compromised?

A breach would not only be bad for our brand integrity, it is a potentially terminal problem. Issuers may revoke our license to accept payment through their means, banks may refuse or revoke our merchant accounts. Even if they don’t we may be forced to notify all our customers of the security breach, and that their data may be compromised, costly and damaging to the brand.

To combat the hackers we would need people smarter than they are, working on ways to ensure that the system remains secure. To put in place and maintain appropriate encryption and indirection to make it impossible to have unauthorized access to the payment records. These people could well be better spent on ways to drive the business forward rather than spending their time build better walls and ditches in the existing system.

Summary

All of the above can lead to quite a significant cost and overhead on our business, and has certainly put me off the notion of creating my own payment collection system in any project. The talk went on to cover Cybersource’s solution to these problems, but this is well documented on their own website and literature, so would be better explained by them directly.

Overall it was quite an interesting talk, hopefully they will follow it up with some discussions around how they have put together a system to handle such large volumes of data.

But back to the importance of the overwrite-worfklow element in identity-ui-configuration.xml. By default when you first add a new process xml file to the /jboss-portal-ha.sar/portal-identity.sar/con/processes directory, it will be picked up by the portal server on its next restart and loaded into its jbpm database. This can lead you to believe that if the portal server is smart enough to recognize a new process has been deployed then it will be smart enough to recognize that the process has been updated too. Therefore simply overwriting the process definition file will upload a new definition to the database right?

Wrong. Unfortunately the server can only detect new process definition files in that directory, and won’t examine existing ones to look for changes. To have the portal server update existing processes that have already been deployed you must set overwrite-workflow to true. However as described in the documentation overwrite-workflow does not function just quite how you might like.

As it states, it overwrites existing process definitions, but that means that it will deploy new versions of all the process definitions that exist rather than examining them and determining which of them really need to be updated. So even if your process definition has not changed, it will deploy a new version.

Of course this may not matter as jbpm versions all the process definitions in the database, and any suspended or inflight process instances continue to run using the version of the process definition that they started with. Production instances of portal are (hopefully) rarely restarted and therefore will rarely deploy new process instances. But there are a couple of gotchas to watch out for.

Portal is frequently restarted during the development phase, often several times a day. This can lead to the jbpm database quickly having many versions of all the process definitions, which in turn can lead to some interesting problems trying to debug issues as it may not be possible to easily relate a process instance to a process definition xml file. This problem will be worsened if the portal database is shared between several developers.

In production, Portal is often clustered. If overwrite-workflow is set to true on all the portal instances then new definition versions will be deployed when each of the portal instances are restarted. This could quickly become a problem if there are many instances or if the portal server needs to be restarted outside of planned maintenance windows. In production there could well be a large number of running process instances that may need to be related back to a particular version for diagnostic purposes.

What can be done?

The simplest solution is to set overwrite-workflow to true only when you actually have new versions of the process definition files to deploy. The downsides of this approach are that it will deploy new versions of all the processes found in the directory, regardless of whether or not they have changed, and that you must remember to keep toggling the switch on and off.

For clustered production nodes, I recommend that during an upgrade process one instance is taken down and used to update the process definitions. After which the setting is immediately set to false again. The same configuration file can be used on all the instances, as its only ever modified briefly during the upgrade of one portal server instance. It should be noted though that if your new definition requires some updated code that you are also deploying, you may well need to take down all of the instances.

Summary

In brief, the original problem was that JBoss portal does not pick up updated instances of jbpm process definitions in the /jboss-portal-ha.sar/portal-identity.sar/conf/processes directory. It will only detect process definition files there the first time they are copied there. Subsequent updates to those files will not be deployed.

A workaround/correction to this issue is to set overwrite-workflow to true in identity-ui-configuration.xml. However this has the side-effects of redeploying all the process definitions in the directory, and continuing to redeploy them each time the portal is restarted, potentially leading to a large number of process definition versions in the database.