Debugging VisiData¶
Not everything goes according to plan. To oversimplify somewhat, you may encounter three sources of errors in VisiData:
- Expected errors, the result of doing something you know will fail in certain situations.
- User-generated errors, the result of issuing commands that are invalid in some way.
- Bugs, which stem from problems in VisiData’s own source code.
These errors can occur in two different scopes:
- Cell errors pertain to specific, individual cells. These are often (but not always) expected errors.
- General errors relate not to the data itself, but to broader usage patterns. These are often (but not always) user-generated errors.
This chapter describes how to investigate errors at both scopes. A quick guide:
| Keystroke(s) | Action |
|---|---|
| Control-e | Examine the latest general error |
| g + Control-e | Examine all general errors encountered |
| z + Control-e | Examine the cell-specific error under your cursor |
Examining cell errors¶
In the Understanding Columns chapter, we’ve already seen an example of what a cell-specific error looks like, when we told VisiData that the HEIGHT column values of the wildlife-strikes dataset should be integers (using #):
File Edit View Column Row Data Plot System Help | VisiData 3.0.2 | Alt+H for help men <STATE │ AIRPORT │ PHASE_OF_FLT │ HEIGHT#│ SPEED │ SPECIES │ BIRDS_STRUCK │ > FL │ VERO BEACH MUNICIP…│ APPROACH │ !│ │ Unknown bird │ 1 │ N AK │ KENAI MUNICIPAL AR…│ APPROACH │ !│ │ Unknown bird │ 1 │ N TX │ DAVID WAYNE HOOKS …│ │ !│ │ Unknown bird │ 1 │ N MO │ LAMBERT-ST LOUIS I…│ APPROACH │ !│ │ Unknown bird │ 1 │ N FL │ POMPANO BEACH AIRP…│ LANDING ROLL │ 0 │ │ Unknown bird │ 1 │ VI │ HENRY E ROHLSEN AR…│ │ !│ │ Unknown bird │ 1 │ O TX │ SAN ANTONIO INTL │ APPROACH │ !│ │ Unknown bird │ 1 │ N TX │ LONE STAR EXECUTIV…│ DEPARTURE │ !│ │ Unknown bird │ 1 │ P FL │ TAMPA INTL │ APPROACH │ 6000 │ │ Unknown bird │ 1 │ N MO │ LAMBERT-ST LOUIS I…│ APPROACH │ !│ │ Owls │ 1 │ N FL │ OPA-LOCKA EXECUTIV…│ APPROACH │ !│ │ Hawks │ 1 │ N CA │ NORMAN Y. MINETA S…│ │ !│ │ Gulls │ 1 │ N FL │ FORT LAUDERDALE/HO…│ APPROACH │ 1500 │ │ Unknown bird - sma…│ 1 │ N AR │ FORT SMITH REGIONA…│ CLIMB │ !│ │ Unknown bird - sma…│ 1 │ N AR │ BILL AND HILLARY …│ LANDING ROLL │ 0 │ │ Unknown bird - sma…│ 1 │ N │ UNKNOWN │ En Route │ !│ │ Unknown bird │ 1 │ P CA │ METRO OAKLAND INTL │ │ !│ │ Unknown bird │ 1 │ N UT │ SALT LAKE CITY INTL│ │ !│ │ Unknown bird │ 1 │ N TX │ GEORGE BUSH INTERC…│ CLIMB │ !│ │ Unknown bird │ 1 │ N FL │ ORLANDO SANFORD IN…│ APPROACH │ !│ │ Unknown bird │ 1 │ N IL │ CHICAGO O'HARE INT…│ CLIMB │ 12000 │ │ Unknown bird │ 1 │ P 1› faa-wildlife-strikes| # type-int 73448 rows
VisiData obeyed but marked some cells with a ! error/warning annotation — specifically the empty cells, because Python considers it impossible to convert nothingness a whole number.
In some cases, we can intuit why a cell is marked with a !. In other cases, we might want to know more. To do so, navigate to the cell and type z + Control-e. If we do this on the second cell of the HEIGHT column, here’s what we’ll see:
File Edit View Column Row Data Plot System Help | VisiData 3.0.2 | Alt+H for help men text ║ Traceback (most recent call last): ║ File "lib/python3.10/site-packages/visidata/wrappers.py", line 105, in wrapply ║ return func(*args, **kwargs) ║ ValueError: invalid literal for int() with base 10: '' ║ zCtrl+E› faa-wildlife-strikes_cell_error| zCtrl+E 4 lines
If you use Python frequently, this output may look familiar; it’s the “stack traceback” for the error that VisiData encountered while trying to convert the blank cell (an empty string) to an integer. If you don’t use Python frequently or at all, you may still be able to get a sense of the issue at hand; the final line is usually the most descriptive.
The error sheet is like any other sheet in VisiData, so you can quit it and return to the previous sheet by pressing q.
Note
There are, in fact, two types of cell errors:
- Errors caused by type conversions, as demonstrated above. VisiData flags these with a yellow exclamation mark.
- Errors stemming from dynamic column expressions, such as
=HEIGHT / 0. VisiData flags these with a red exclamation mark.
Examining general errors¶
Let’s try creating a general error on purpose. Let’s press = to create a new column, and type ( at the prompt, and then Enter. We’ll be greeted by an error message at the bottom of the screen:
DELTA AIR LINES │ A-320 │ 05/07/15 00:00:00 │ CA │ METRO OAKLAND INTL │ │ DELTA AIR LINES │ A-320 │ 05/08/15 00:00:┌────────────────────────────────| statuses |─┐│ LUFTHANSA │ A-380 │ 05/10/15 00:00:│ SyntaxError: '(' was never closed (<expr>, ││ BUSINESS │ C-172 │ 05/08/15 00:00:│ line 1) ││ SPIRIT AIRLINES │ A-319 │ 05/10/15 00:00:└─────────────────────────────────────────────┘│ 1› faa-wildlife-strikes| = addcol-expr 73448 rows
Why’s that? In Python, all open parentheses need to be closed.
To examine this general error in greater detail, we can press Control-e — now, or at any point until you trigger another error — to reveal its full traceback in a VisiData text sheet:
File Edit View Column Row Data Plot System Help | VisiData 3.0.2 | Alt+H for help men text ║ Traceback (most recent call last): ║ File "lib/python3.10/site-packages/visidata/basesheet.py", line 211, in execCommand ║ escaped = super().execCommand2(cmd, vdglobals=vdglobals) ║ File "lib/python3.10/site-packages/visidata/extensible.py", line 79, in wrappedfunc ║ r = oldfunc(*args, **kwargs) ║ File "lib/python3.10/site-packages/visidata/basesheet.py", line 76, in execCommand2 ║ exec(code, vdglobals, LazyChainMap(vd, self)) ║ File "addcol-expr", line 1, in <module> ║ 'VisiData: a curses interface for exploring and arranging tabular data' ║ File "lib/python3.10/site-packages/visidata/column.py", line 544, in __init__ ║ self.expr = expr or name ║ File "lib/python3.10/site-packages/visidata/column.py", line 570, in expr ║ self.compiledExpr = compile(expr, '<expr>', 'eval') if expr else None ║ File "<expr>", line 1 ║ ( ║ ^ ║ SyntaxError: '(' was never closed ║ Ctrl+E› errors_recent| Shift+G go-bottom 17 lines
Tip
You can type g + Control-e to open a sheet listing all general errors you have encountered during your current VisiData session.
How to report a bug¶
If you encounter an error that seems to come from a bug within VisiData itself (rather than an invalid command), follow VisiData’s “Help and Support” instructions. The developers ask that bug reports include a “command log” that reproduces the issue, so that they can investigate it with more precision. The next chapter demonstrates how to do so.