I have an eCommerce platform that creates a basket as soon as the user lands on the site if they don’t already have one (from session or cookies). It then stores the basket information in session and in cookies to remember what the user has added to their basket for a month.
The problem is if they aren’t capable of having a session or cookies, such as a bot, then it creates a basket every time they visit a page. This means a bot crawling through the website can easily create hundreds of baskets.
One of the options we’re exploring is changing our code so that a basket is only created when a user adds an item, and so far it seems the best option, but it’s also the most time consuming option as a lot of the code base assumes a basket exists at all times. That would need to be changed so that it checks for a basket and handles a basket not existing. We’d like to find a less intensive solution if possible.
Another option we’ve explored is periodically clearing out the table in the database of all old baskets to help mitigate the issue. However this is just mitigating, and not solving the issue. I’d like to solve the problem if I can.
How can I identify if the user is incapable of having a session and cookies so that I can stop the basket from being created? Or is there a better way of dealing with bots in this instance?
4
Any agent that isn’t capable of handling cookies or even sessions will probably also be unable to tell you reliably that it doesn’t handle them correctly.
However, indexing bots that crawl arbitrary pages will usually specify a characteristic user-agent string and/or obey your robots.txt rules, so you should deal with them via one of those two well-established bot-handling protocols.
I would recommend doing some detection based on the users user-agent. You can use a library like UA Detector to identify users as bots and trigger different backend behavior. You will likely need to also take the approach recommended by Mike in the comments to your question as your traffic continues to grow and you start getting hit by unidentified bots.
3