Debug a failed extractor execution

How to debug a failed execution

Debugging web scraping robots has always been tedious and time-consuming. At dexi.io, we make this a lot easier by providing four valuable tools for finding and resolving errors:

• Screenshots
• Error messages
• Debug mode
• Execution logs

Screenshots

Every execution gets a screenshot, whether the execution was a failure or a success, depending on the option you choose. The screenshot will be taken immediately before the execution stops, so you will get only one screenshot for an execution with 0 - 1 inputs.

If you have additional input rows, you will get one screenshot per input row. You can view all screenshots from an execution by selecting the View link in the Screenshots column. The number enclosed in parentheses indicates how many screenshots are available.

The error message

Failed executions feature a red triangle icon. Hovering your mouse cursor over this icon reveals if there has been an error, and selecting the Log button will download a log file containing more information.

Debug mode

To open your robot in Debug Mode, select the small green bug icon for the failed execution.

This opens the robot editor to display all the steps of the execution that failed, even if the error mode is set to Ignore or Skip.

The Web page is displayed in the exact same state it was in at the time it failed. You can see how the page looked when it failed, along with which loop iteration and branch the robot was processing at the time.

Debug Mode is a snapshot of how the page looked at that time, so you cannot continue along the timeline from that point. It’s intended as a visual replica of the page state to help you identify and fix the error causing the failure. You can then restart the robot to verify that you solved the problem.

To restart the robot, simply select the Restart button.

The execution log

The execution log can be used if none of the above methods prove useful. It's a last resort because it's not exactly easy, and it can take a while to make sense of.

Understanding execution logs
These are used when something goes wrong and is either failing completely or not getting the data you expected. A last resort for debugging is the execution logs.

To open a log for a failed execution:

1. Go to the results page.
2. Select the log icon.

This prompts you to download a log file. This is very technical but most of the time you will be able to see which specific step caused your robot to fail, and the reason for it.

If you do not understand it, our technical support staff are more than happy to help, and the log file is an invaluable tool to help us resolve issues.