History
We are currently using a so called redirect model for our online payments (where you send the payer to a payment gateway, where he inputs his payment details – the gateway will then return him to a success/failure callback page). That’s easy and straight-forward, but unfortunately quite inconvenient and at times confusing for our customers (leaving the site, changing their credit card details with an additional login on another site etc).
Intention & Problem description
We are now intending to switch to an integrated approach using an exchange of XML requests and responses. My problem is on how to cater with all (or rather most) of the things that may happen during processing – bearing in mind that normally simplicity is robust whereas complexity is fragile.
Examples
- User abort: The user inputs Credit Card details and hits submit. An XML message to the provider’s gateway is sent and waiting for response. The user hits “stop” in his browser or closes the window.
- ignore_user_abort() in PHP may be an option – but is that reliable?
- might it be better to redirect the user to a “please wait”-page, that in turn opens an AJAX or other request to the actual processor that does not rely on the connection?
- Database goes away
- sounds over-complicated, but with e.g. a webserver in the States and a DB in the UK, it has happened and will happen again: User clicks together his order, payment request has been sent to the provider but the response cannot be stored in the database. What approach could I use, using PHP to sort of start an SQL like “Transaction” that only at the very end gets committed or rolled back, depending on the individual steps? Should then neither commit or roll back have happened, I could sort of “lock” the user to prevent him from paying again or to improperly account for payments – but how?
- And what else do I need to consider technically? None of the integration examples of e.g. Worldpay, Realex or SagePay offer any insight, and either my search engine or my search terms weren’t good enough to find somebody else’s thoughts on this.
Thank you very much for any insight on how you would approach this!
4
As a developer working almost exclusively with payments I perhaps can give you some pointers about where the possible pitfalls are. I’m working for a site with high traffic, ~40 active payment service providers (PSPs) and dealing with tens of thousands transactions per day. And trust me, the sh*t will always hit the fan and all you can do is prepare as much as possible for dealing with the situation from that point.
Log, log and log some more…
This is the most important part in your payment process, make sure you have a record of everything. Ask yourself the questions “could this piece of information help me once in a million transactions?”. If the answer is yes, log it!
Our setup is that the main point of logging is the database. Every initiated, failed, settled, redirected, etc transaction is stored per PSP in tables. If the PSP uses an API, all API-calls are logged (both requests sent and responses). All callbacks are logged in tables as well. And so forth..
When we encounter an unexpected event or exception we log it to the PHP log file with a certain format to make it searchable and easily found based on transaction ID, user ID, etc.
You will be grateful that you have all the data if you get sued for example.
Monitoring and alerts
Build in monitoring logic with some simple tests that will alert you through email when something goes wrong. For example, if 3 callbacks in a row fails for a PSP. All of these small things that can make you work on resolving issues quickly instead of reacting to them after your customers have made you aware of them (which doesn’t always happen) is very important.
Database transactions
If you have the luxury of setting up or changing your database structure to support database transactions, for example use InnoDB on MySQL. Take a look at this answer on how to handle it in the code. You should make sure that every step of a transaction is completed or none is, having partially completed transactions is just a burden on you and your system. For example: 1) User completes transaction, 2) User gets rewarded somehow. If not both step 1 and 2 are met here, the transaction should not be set as complete in step 1.
Technical people to contact
When you have a technical issue, make sure you have someone technical to contact immediately. Having an account manager is pretty useless, especially if that person decides to mediate between you and their technician.
Live world example that happened to me: one PSP all of the sudden stops working, nothing in our code had changed and they say they haven’t changed anything as well. After sending debug information back and forth with them (through their account manager), one of their technicians catches that the WSDL-schema fetches seems off. Then they realize they updated their WSDL-schema and after some debugging I realize that PHP cached the old WSDL-schema. This could’ve been caught sooner as resolved if I just had a technical person on Skype to contact directly. Or if they just admitted to changing their WSDL-schema, but it’s often hard to get a confession out of the PSPs when something goes wrong..
Final words…
Prepare all you can and assume that ANYTHING can happen 🙂
Not sure this was exactly the kind of answer you were after, but I’m trying to provide a real world example of payments and how you can prevent the biggest traps and pitfalls. Let me know if there’s something you would like me to elaborate on or if you’re interested in another topic.
2
In my honest opinion, the best way to to manage transactions through a Payment Gateway is through web services. A number of different payment gateways offer these kinds of services for competitive flat percentage rates per transaction + the typical credit card commission that the customers credit card will take as well (Eg. Cybersource). With much everything in life, you get what you pay for, a 9 cent/transaction doesn’t exist with a reputable payment processor that doesn’t force you to redirect users.
Tokenization and secure data
Storing credit card details on your system is just asking for trouble, and depending on where you live there may even be legal requirements and possible criminal laws against how this data is being managed. Unless you are Amazon, you shouldn’t have to worry about all that technical and legal complexity. The payment processor takes this risk and complexity away from you by having SSL encrypted web services where credit card details can be passed through your site to your server, and from your server to the payment processor web services to submit a payment through your account with the processor. In return, the web service will give you back an authorization code, or a token that uniquely signifies the transaction on their system, and gives you a reference number for quickly searching through your account for specific transactions. You can retain the token in your database and it is of no real consequence to the users if the tokens are compromised on your system. Further by not storing credit card details at all on your system, users bank accounts are safe even if your system has been compromised.
Retention of payment records
This approach to integrating with a payment processor has the added benefit of being able to maintain your own payment and purchase records that are seperate from that of your payment processor, yet linked through the token. This allows you to run revenue reports through your system, analytical queries and users can view their purchases without actually having to query the payment processor for information.
PCI Compliance
Any reputable payment processor will have all of their systems and data hosted in PCI Compliant servers and facilities. For more information on the details of PCI Compliance itself, and why it is important, see this site.
There are certain requirements that they must adhere to, their data center must be secured physically as well as technically. They cannot have any unencrypted WAP’s on their network, etc… None of this will really concern you however there are a number of requirements expected of merchant systems for them to claim PCI Compliance as well. Some of these requirements include but are not limited to:
-
Using SSL encryption on any payment request through your site.
-
Under no circumstances should there be credit card numbers stored anywhere in any of your systems, including log files and even serialized objects.
-
Application servers communicating with the payment processor must be behind a well configured firewall that only allows the minimum number of open ports required.
-
Only appropriate technical personnel should be allowed physical or remote access to these application servers on your network.
-
Others…
Typically most payment processors will provide an online tool that can run on your network and identify possible issues with becoming PCI Compliant. It is still not a NECESSARY or REQUIRED thing to have to work with a payment processor, but the payment processor reserves the right to charge you additional fees if you utilize their services and do not meet PCI Compliance standards.
2
Thanks to ideas from previous answers, I’d like to propose the solution I have now though of; I hope this may invite one or two comments from the cyclopedical knowledge present here and help me to improve it.
Just for clarification: Although important issues, my question was not dealing with how to choose a Payment Service Provider (PSP), PCI DSS requirements or legal implications of anything, but just technical pitfalls one might stumble across when implementing a “direct” instead of a “redirect” model with any PSP.
Prerequisites
The customer is at the stage where he inputs credit card details into your form. I know what he owes and what amount and currency I want to charge to the credit card he submits.
Solution (?)
- On submit of the form containing the Credit Card details, I will use Javascript to stop the default action on the submit button (which, if JS or AJAX weren’t available, could go to an error page telling the user to get a more recent browser…)
- The (repeated by a timer) AJAX request would call a status update script, that in itself triggers the actual processing in a seperate file; using “ignore_user_abort() should the client close the browser or something
- so AJAX calls e.g. the method ->StatusUpdate() – if called for the first time, it would start the process and return “pleasewait”, on subsequent calls it would evaluate a session variable or something to update the status
- I would update the user’s record in the database to set a “lockSession” flag to “locked” – should something happen before unlock, the user would be told on subsequent logins to contact the office to sort out any mess that may have happend…
- insert a payment record into the DB with a “prepared” status
- Submit the request to the PSP, using the order id of the prepared payment record. Depending on the PSP, the request can be submitted as a delayed transaction (meaning it won’t be settled automatically)
- update the prepared record with it’s new status (if declined, unlock session and go back to the payment page; if successful, update with status “transmitted”)
- If it was successful, we would not have the auth reference and other important details from the PSP – so send “settle” request to the PSP for the obtained auth reference.
- If successful, update the payment record to completed (or in my case, move and update it from PendingTransactions to PaymentTransactions)
- Remove the “lock” flag from the user’s record
Other considerations
Please understand the above as an invitation to share your knowledge and ideas – they’re only ideas so far!
Logging
I would very much agree with nerdklers on the logging – I’m currently considering to use some form of file-based logging for every stage of the above process (and to keep logs for a few days even if complete and verified – who knows…)
Database transactions
I’m not sure I would agree with using database transactions. We’re using MS SQL, and with default settings, any transaction would be rolled back on loss of connection. And when setting the transaction to not auto-commit or auto-rollback, one might end up with a few open DB transactions and some implications on record/table locking etc.
I think I’ll prefer multiple updates in the process; should any of those fail (and the process consequently be terminated) the user would still be prevented from re-submitting a possibly completed (but not yet recorded) transaction until it has been resolved manually.
AJAX, political
I’m not entirely certain whether we still need to consider browsers that won’t support AJAX. I guess that’s a “political” topic anyway. I think we could afford to take liberties in our business, others may possibly not.
AJAX, technical
I’m not so sure about my approach with calling a “StatusUpdate” method asynchronously yet, that will on first call attempt to trigger the actual process. But I shall make this a seperate question should I not find good examples – which I guess I might well find. Anyhow, if you know a best-practice please do let me (and others) know in your comment.
Final thoughts
It’d be nice if I had a thought even remotely approaching the term “final” – this if work in progress for me and I am much obliged for any feedback and further hints on how this could be improved.
3