Understanding Columns

Columns in VisiData behave much (though not entirely) like they do in spreadsheet programs and statistical computing environments. Getting a solid grasp of them will put you well on your way to mastering VisiData.

Basic column attributes

Every column in VisiData has the following basic properties, all of which can be modified:

  • Name: Self explanatory.
  • Width: The number of characters the column occupies on the screen.
  • Type: The kind of data — text, integer, float, currency, or date — that the column will be interpreted as.

We’ll get to a few other properties later.

How to view all columns and their attributes

To see all the columns in your dataset, press Shift-C. This will open the “Columns Sheet,” which lists each column and its attributes. For the FAA dataset we’ve been using, it should look like this:

  File  Edit  View  Column  Row  Data  Plot  System  Help     | VisiData 3.0.2 | Alt+H for help men
 name               width# type  fmtstr  value               aggregators                     
 OPERATOR          ║    20 │      │        │ BUSINESS           │             ║                     
 ATYPE                 14                PA-28                                                
 INCIDENT_DATE         19                05/22/15 00:00:00                                    
 STATE                  7                FL                                                   
 AIRPORT               20                VERO BEACH MUNICIP…                                  
 PHASE_OF_FLT          14                APPROACH                                             
 HEIGHT                                                                                      
 SPEED                                                                                       
 SPECIES                                Unknown bird                                         
 BIRDS_STRUCK                           1                                                    
 EFFECT                                 NONE                                                 
 DAMAGE                                 M                                                    
 COST_REPAIRS                                                                                
 PERSON                                 Tower                                                
 REMAINS_COLLECTED                      0                                                    
 REMARKS                                N9240F was right b…                                  





Shift+C› faa-wildlife-strikes_columns|                               Shift+C            16 columns 

The Columns Sheet is handy for quickly getting a sense of your dataset’s structure. You can navigate the sheet as if it were just another dataset. And just like you can do with a standard data sheet, you can leave this column sheet by pressing q.

Note

The Columns Sheet is one of several “metasheets” in VisiData; the Sheets Sheet from the previous chapter is another example. You’ll encounter a few other metasheets in this tutorial.

How to set column types

Rather than guess at your column’s data types, VisiData assumes that they’re all plain-old text.

If a column of really is just text, then great.

But if that column is a number or date, and you want to do any math-y operations on your column (e.g., sorting, summing, averaging, et cetera), you’ll have to specify its type.

To set a column’s type, navigate over to that column and press one of the following keys:

Keystroke Column type Examples
# Integer 0, -1, 5000000
% Float 0.5, -3.14, 23.45557
$ Currency $4.99, €20, 100 100
@ Date 2018-04-06, April 6, 2018, 04/06/2018
~ Text anything!

When you do so, the corresponding symbol (e.g., $ for currency) will appear in the column’s heading.

For instance, here’s what you should see if you navigate to the wildlife-strike database’s HEIGHT column, and then press # to tell VisiData that the height values are integers:

  File  Edit  View  Column  Row  Data  Plot  System  Help     | VisiData 3.0.2 | Alt+H for help men
<STATE  AIRPORT             PHASE_OF_FLT  HEIGHT# SPEED  SPECIES             BIRDS_STRUCK  >
 FL    │ VERO BEACH MUNICIP…│ APPROACH     │       !│       │ Unknown bird       │ 1            │ N 
 AK     KENAI MUNICIPAL AR… APPROACH            !        Unknown bird        1             N 
 TX     DAVID WAYNE HOOKS …                     !        Unknown bird        1             N 
 MO     LAMBERT-ST LOUIS I… APPROACH            !        Unknown bird        1             N 
 FL     POMPANO BEACH AIRP… LANDING ROLL       0         Unknown bird        1               
 VI     HENRY E ROHLSEN AR…                     !        Unknown bird        1             O 
 TX     SAN ANTONIO INTL    APPROACH            !        Unknown bird        1             N 
 TX     LONE STAR EXECUTIV… DEPARTURE           !        Unknown bird        1             P 
 FL     TAMPA INTL          APPROACH        6000         Unknown bird        1             N 
 MO     LAMBERT-ST LOUIS I… APPROACH            !        Owls                1             N 
 FL     OPA-LOCKA EXECUTIV… APPROACH            !        Hawks               1             N 
 CA     NORMAN Y. MINETA S…                     !        Gulls               1             N 
 FL     FORT LAUDERDALE/HO… APPROACH        1500         Unknown bird - sma… 1             N 
 AR     FORT SMITH REGIONA… CLIMB               !        Unknown bird - sma… 1             N 
 AR     BILL AND  HILLARY … LANDING ROLL       0         Unknown bird - sma… 1             N 
        UNKNOWN             En Route            !        Unknown bird        1             P 
 CA     METRO OAKLAND INTL                      !        Unknown bird        1             N 
 UT     SALT LAKE CITY INTL                     !        Unknown bird        1             N 
 TX     GEORGE BUSH INTERC… CLIMB               !        Unknown bird        1             N 
 FL     ORLANDO SANFORD IN… APPROACH            !        Unknown bird        1             N 
 IL     CHICAGO O'HARE INT… CLIMB          12000         Unknown bird        1             P 
1› faa-wildlife-strikes|                                              #   type-int      73448 rows 

Note

As you can see above, if a cell cannot be converted into the type you’ve assigned it, VisiData will display ! in the right-side margin of that cell.

To entirely remove a column’s assigned type, navigate to the column and type z~.

How to rename columns

  • Navigate to the column that you want to rename:
  File  Edit  View  Column  Row  Data  Plot  System  Help     | VisiData 3.0.2 | Alt+H for help men
 OPERATOR            ATYPE         INCIDENT_DATE      STATE  AIRPORT             PHASE_OF_FLT>
 BUSINESS           │ PA-28        │ 05/22/15 00:00:00 │ FL    │ VERO BEACH MUNICIP…│ APPROACH     │
 BUSINESS            BE-1900       06/18/15 00:00:00  AK     KENAI MUNICIPAL AR… APPROACH     
  • Press ^, which enters column-name-editing mode (evident by the underscores and change in background highlighting):
  File  Edit  View  Column  Row  Data  Plot  System  Help     | VisiData 3.0.2 | Alt+H for help men
 OPERATOR           ATYPE________   INCIDENT_DATE      STATE  AIRPORT             PHASE_OF_FLT>
 BUSINESS           │ PA-28        │ 05/22/15 00:00:00 │ FL    │ VERO BEACH MUNICIP…│ APPROACH     │
 BUSINESS            BE-1900       06/18/15 00:00:00  AK     KENAI MUNICIPAL AR… APPROACH     
  • Then, type what you want the column to be renamed:
  File  Edit  View  Column  Row  Data  Plot  System  Help     | VisiData 3.0.2 | Alt+H for help men
 OPERATOR           Aircraft_____   INCIDENT_DATE      STATE  AIRPORT             PHASE_OF_FLT>
 BUSINESS           │ PA-28        │ 05/22/15 00:00:00 │ FL    │ VERO BEACH MUNICIP…│ APPROACH     │
 BUSINESS            BE-1900       06/18/15 00:00:00  AK     KENAI MUNICIPAL AR… APPROACH     
  • And then press Enter to complete the process:
  File  Edit  View  Column  Row  Data  Plot  System  Help     | VisiData 3.0.2 | Alt+H for help men
 OPERATOR            Aircraft      INCIDENT_DATE      STATE  AIRPORT             PHASE_OF_FLT>
 BUSINESS           │ PA-28        │ 05/22/15 00:00:00 │ FL    │ VERO BEACH MUNICIP…│ APPROACH     │
 BUSINESS            BE-1900       06/18/15 00:00:00  AK     KENAI MUNICIPAL AR… APPROACH     

How to expand, shrink, and remove columns

When you load a dataset, VisiData will try to choose reasonable widths for your columns. You can adjust them in a few ways:

Keystroke(s) Action
_ Expands the width of current column to fit text in all visible rows
g_ Expands the width of all columns to fit text in all visible rows
z_ + n Sets the current column’s width to n characters
- Hides the current column by setting its width to 0
gv Unhides all columns
z- Shrinks the current column’s width in half

Note

What’s the deal with “z”?

Much like with g, you’ll notice that many VisiData commands can be prefixed with z. The effect is typically to narrow or specify the scope of the action; e.g., - hides a column entirely, while z- only shrinks it to half-width.

How to move columns’ positions

Sometimes you want to view a dataset’s columns in a different order than they appear in the dataset. To do that in VisiData, use the following keystrokes:

Keystroke(s) Action
Shift-H Moves column one position to the left
Shift-L Moves column one position to the right

Tip

Similarly, you can use Shift-J to move a row down one position, and Shift-K to move a row up one position.

How to designate “key” columns

For any sheet, you can designate any number of columns as “key” columns. They serve two functions:

  • They stay pinned to the left-hand side of the sheet when you scroll horizontally.
  • They get special treatment for certain commands, such as when joining sheets. (More on this later.)

To turn a column into a key column (or vice-versa), navigate to that column and press !.

For example, say we’ve navigated to the AIRPORT column of the FAA dataset. Pressing ! will turn this:

  File  Edit  View  Column  Row  Data  Plot  System  Help     | VisiData 3.0.2 | Alt+H for help men
 OPERATOR            Aircraft      INCIDENT_DATE      STATE  AIRPORT             PHASE_OF_FLT>
 BUSINESS           │ PA-28        │ 05/22/15 00:00:00 │ FL    │ VERO BEACH MUNICIP…│ APPROACH     │
 BUSINESS            BE-1900       06/18/15 00:00:00  AK     KENAI MUNICIPAL AR… APPROACH      BUSINESS            PA-46 MALIBU  09/20/15 00:00:00  TX     DAVID WAYNE HOOKS …               DELTA AIR LINES     B-717-200     11/07/15 00:00:00  MO     LAMBERT-ST LOUIS I… APPROACH      BUSINESS            BE-90 KING    12/17/15 00:00:00  FL     POMPANO BEACH AIRP… LANDING ROLL  DELTA AIR LINES     B-757         07/17/15 00:00:00  VI     HENRY E ROHLSEN AR…               DELTA AIR LINES     B-717-200     08/02/15 00:00:00  TX     SAN ANTONIO INTL    APPROACH      BUSINESS            C-414         08/03/15 00:00:00  TX     LONE STAR EXECUTIV… DEPARTURE     ALLEGIANT AIR       MD-80         09/02/15 00:00:00  FL     TAMPA INTL          APPROACH      TRANS STATES AIRLI… EMB-145       09/07/15 00:00:00  MO     LAMBERT-ST LOUIS I… APPROACH      BUSINESS            C-172         11/28/15 00:00:00  FL     OPA-LOCKA EXECUTIV… APPROACH      GOVERNMENT          EC120         12/08/15 00:00:00  CA     NORMAN Y. MINETA S…               AMERICAN AIRLINES   A-321         05/06/15 00:00:00  FL     FORT LAUDERDALE/HO… APPROACH      EXPRESSJET AIRLINES CRJ100/200    05/06/15 00:00:00  AR     FORT SMITH REGIONA… CLIMB         MESA AIRLINES       CRJ900        05/08/15 00:00:00  AR     BILL AND  HILLARY … LANDING ROLL  BUSINESS            HELICOPTER    05/06/15 00:00:00         UNKNOWN             En Route      DELTA AIR LINES     A-320         05/07/15 00:00:00  CA     METRO OAKLAND INTL                DELTA AIR LINES     A-320         05/08/15 00:00:00  UT     SALT LAKE CITY INTL               LUFTHANSA           A-380         05/10/15 00:00:00  TX     GEORGE BUSH INTERC… CLIMB         BUSINESS            C-172         05/08/15 00:00:00  FL     ORLANDO SANFORD IN… APPROACH      SPIRIT AIRLINES     A-319         05/10/15 00:00:00  IL     CHICAGO O'HARE INT… CLIMB        1› faa-wildlife-strikes|                                          c   go-col-regex      73448 rows 

… into this:

  File  Edit  View  Column  Row  Data  Plot  System  Help     | VisiData 3.0.2 | Alt+H for help men
 AIRPORT             OPERATOR            Aircraft      INCIDENT_DATE      STATE  PHASE_OF_FLT>
 VERO BEACH MUNICIP…║ BUSINESS           │ PA-28        │ 05/22/15 00:00:00 │ FL    │ APPROACH     │
 KENAI MUNICIPAL AR… BUSINESS            BE-1900       06/18/15 00:00:00  AK     APPROACH      DAVID WAYNE HOOKS … BUSINESS            PA-46 MALIBU  09/20/15 00:00:00  TX                   LAMBERT-ST LOUIS I… DELTA AIR LINES     B-717-200     11/07/15 00:00:00  MO     APPROACH      POMPANO BEACH AIRP… BUSINESS            BE-90 KING    12/17/15 00:00:00  FL     LANDING ROLL  HENRY E ROHLSEN AR… DELTA AIR LINES     B-757         07/17/15 00:00:00  VI                   SAN ANTONIO INTL    DELTA AIR LINES     B-717-200     08/02/15 00:00:00  TX     APPROACH      LONE STAR EXECUTIV… BUSINESS            C-414         08/03/15 00:00:00  TX     DEPARTURE     TAMPA INTL          ALLEGIANT AIR       MD-80         09/02/15 00:00:00  FL     APPROACH      LAMBERT-ST LOUIS I… TRANS STATES AIRLI… EMB-145       09/07/15 00:00:00  MO     APPROACH      OPA-LOCKA EXECUTIV… BUSINESS            C-172         11/28/15 00:00:00  FL     APPROACH      NORMAN Y. MINETA S… GOVERNMENT          EC120         12/08/15 00:00:00  CA                   FORT LAUDERDALE/HO… AMERICAN AIRLINES   A-321         05/06/15 00:00:00  FL     APPROACH      FORT SMITH REGIONA… EXPRESSJET AIRLINES CRJ100/200    05/06/15 00:00:00  AR     CLIMB         BILL AND  HILLARY … MESA AIRLINES       CRJ900        05/08/15 00:00:00  AR     LANDING ROLL  UNKNOWN             BUSINESS            HELICOPTER    05/06/15 00:00:00         En Route      METRO OAKLAND INTL  DELTA AIR LINES     A-320         05/07/15 00:00:00  CA                   SALT LAKE CITY INTL DELTA AIR LINES     A-320         05/08/15 00:00:00  UT                   GEORGE BUSH INTERC… LUFTHANSA           A-380         05/10/15 00:00:00  TX     CLIMB         ORLANDO SANFORD IN… BUSINESS            C-172         05/08/15 00:00:00  FL     APPROACH      CHICAGO O'HARE INT… SPIRIT AIRLINES     A-319         05/10/15 00:00:00  IL     CLIMB        1› faa-wildlife-strikes|                                               !   key-col      73448 rows 

Manipulating columns from the Columns Sheet

You can do nearly all of the above from the Columns Sheet (Shift-C). When you’re dealing with datasets with a particularly large number of columns, doing it this way can often be easier; you can see more of the columns at once, and you can adjust multiple columns at once.

  • Moving columns: In the Columns Sheet, each column is represented as a row; you can reposition them using Shift-J and Shift-K.
  • Editing column names: In the Columns Sheet, you can edit each column’s name like you would any other row cell. Just navigate to the name, and press e to start editing.
  • Setting column types: You can select multiple rows of the Columns Sheet and type g$, for example, to set all of the selected columns’ types to currency.
  • Setting column widths: You can edit the width field of the Columns Sheet to adjust any column’s width. You can also give multiple columns to the same width by selecting each of their rows, and typing ge + number, where number is the desired width.

For instance, here’s the Columns Sheet for the FAA dataset before we’ve made any changes:

  File  Edit  View  Column  Row  Data  Plot  System  Help     | VisiData 3.0.2 | Alt+H for help men
 name               width# type  fmtstr  value               aggregators                     
 OPERATOR          ║    20 │      │        │ BUSINESS           │             ║                     
 ATYPE                 14                PA-28                                                
 INCIDENT_DATE         19                05/22/15 00:00:00                                    
 STATE                  7                FL                                                   
 AIRPORT               20                VERO BEACH MUNICIP…                                  
 PHASE_OF_FLT          14                APPROACH                                             
 HEIGHT                                                                                      
 SPEED                                                                                       
 SPECIES                                Unknown bird                                         
 BIRDS_STRUCK                           1                                                    
 EFFECT                                 NONE                                                 
 DAMAGE                                 M                                                    
 COST_REPAIRS                                                                                
 PERSON                                 Tower                                                
 REMAINS_COLLECTED                      0                                                    
 REMARKS                                N9240F was right b…                                  





Shift+C› faa-wildlife-strikes_columns|                               Shift+C            16 columns 

Next, we use the s key to select the four columns we’d like to change:

  File  Edit  View  Column  Row  Data  Plot  System  Help     | VisiData 3.0.2 | Alt+H for help men
 name               width# type  fmtstr  value               aggregators                     
 OPERATOR              20                BUSINESS                                             
 ATYPE                 14                PA-28                                                
 INCIDENT_DATE         19                05/22/15 00:00:00                                    
 STATE                  7                FL                                                   
 AIRPORT               20                VERO BEACH MUNICIP…                                  
 PHASE_OF_FLT          14                APPROACH                                             
HEIGHT            │      │        │                    │             ║                     
SPEED             │      │        │                    │             ║                     
 SPECIES                                Unknown bird                                         
BIRDS_STRUCK      │      │        │ 1                  │             ║                     
 EFFECT                                 NONE                                                 
 DAMAGE                                 M                                                    
COST_REPAIRS      │      │        │                    │             ║                     
 PERSON            │      │        │ Tower              │             ║                     
 REMAINS_COLLECTED                      0                                                    
 REMARKS                                N9240F was right b…                                  





Shift+C› faa-wildlife-strikes_columns|                       s   select-row         16 columns  •4 

We’d like to make them all numeric columns, so we type g#, which results in this:

  File  Edit  View  Column  Row  Data  Plot  System  Help     | VisiData 3.0.2 | Alt+H for help men
 name               width# type  fmtstr  value               aggregators                     
 OPERATOR              20                BUSINESS                                             
 ATYPE                 14                PA-28                                                
 INCIDENT_DATE         19                05/22/15 00:00:00                                    
 STATE                  7                FL                                                   
 AIRPORT               20                VERO BEACH MUNICIP…                                  
 PHASE_OF_FLT          14                APPROACH                                             
HEIGHT            │ int  │        │                    │             ║                     
SPEED             │ int  │        │                    │             ║                     
 SPECIES                                Unknown bird                                         
BIRDS_STRUCK      │ int  │        │ 1                  │             ║                     
 EFFECT                                 NONE                                                 
 DAMAGE                                 M                                                    
COST_REPAIRS      │ int  │        │                    │             ║                     
 PERSON            │      │        │ Tower              │             ║                     
 REMAINS_COLLECTED                      0                                                    
 REMARKS                                N9240F was right b…                                  





Shift+C› faa-wildlife-strikes_columns|               g#   type-int-selected         16 columns  •4 

And we’d like to make them uniformly narrow, so we navigate to the width column and type ge8 + Enter, which gives us this:

  File  Edit  View  Column  Row  Data  Plot  System  Help     | VisiData 3.0.2 | Alt+H for help men
 name               width# type  fmtstr  value               aggregators                     
 OPERATOR              20                BUSINESS                                             
 ATYPE                 14                PA-28                                                
 INCIDENT_DATE         19                05/22/15 00:00:00                                    
 STATE                  7                FL                                                   
 AIRPORT               20                VERO BEACH MUNICIP…                                  
 PHASE_OF_FLT          14                APPROACH                                             
HEIGHT            ║     8 │ int  │        │                    │             ║                     
SPEED             ║     8 │ int  │        │                    │             ║                     
 SPECIES                                Unknown bird                                         
BIRDS_STRUCK      ║     8 │ int  │        │ 1                  │             ║                     
 EFFECT                                 NONE                                                 
 DAMAGE                                 M                                                    
COST_REPAIRS      ║     8 │ int  │        │                    │             ║                     
 PERSON            ║      │      │        │ Tower              │             ║                     
 REMAINS_COLLECTED                      0                                                    
 REMARKS                                N9240F was right b…                                  


                                                       ┌─────────────────────────────| statuses |─┐
                                                       │ set 4 cells to 1 values                                                                         └──────────────────────────────────────────┘
Shift+C› faa-wildlife-strikes_columns|                ge   setcol-input         16 columns  [M] •4 

To see how these changes have affected your data sheet, press q to exit the Columns Sheet, and then navigate over to the HEIGHT column:

  File  Edit  View  Column  Row  Data  Plot  System  Help     | VisiData 3.0.2 | Alt+H for help men
<STATE  AIRPORT             PHASE_OF_FLT  HEIGHT# SPEED # SPECIES             BIRDS_# EFFEC>
 FL    │ VERO BEACH MUNICIP…│ APPROACH     │       !!│ Unknown bird       │      1 │ NONE   
 AK     KENAI MUNICIPAL AR… APPROACH            !       ! Unknown bird             1  NONE   
 TX     DAVID WAYNE HOOKS …                     !       ! Unknown bird             1  NONE   
 MO     LAMBERT-ST LOUIS I… APPROACH            !       ! Unknown bird             1  NONE   
 FL     POMPANO BEACH AIRP… LANDING ROLL       0        ! Unknown bird             1         
 VI     HENRY E ROHLSEN AR…                     !       ! Unknown bird             1  Other  
 TX     SAN ANTONIO INTL    APPROACH            !       ! Unknown bird             1  NONE   
 TX     LONE STAR EXECUTIV… DEPARTURE           !       ! Unknown bird             1  PRECA… 
 FL     TAMPA INTL          APPROACH        6000        ! Unknown bird             1  NONE   
 MO     LAMBERT-ST LOUIS I… APPROACH            !       ! Owls                     1  NONE   
 FL     OPA-LOCKA EXECUTIV… APPROACH            !       ! Hawks                    1  NONE   
 CA     NORMAN Y. MINETA S…                     !       ! Gulls                    1  NONE   
 FL     FORT LAUDERDALE/HO… APPROACH        1500        ! Unknown bird - sma…      1  NONE   
 AR     FORT SMITH REGIONA… CLIMB               !       ! Unknown bird - sma…      1  NONE   
 AR     BILL AND  HILLARY … LANDING ROLL       0        ! Unknown bird - sma…      1  NONE   
        UNKNOWN             En Route            !       ! Unknown bird             1  PRECA… 
 CA     METRO OAKLAND INTL                      !       ! Unknown bird             1  NONE   
 UT     SALT LAKE CITY INTL                     !       ! Unknown bird             1  NONE   
 TX     GEORGE BUSH INTERC… CLIMB               !       ! Unknown bird             1  NONE   
 FL     ORLANDO SANFORD IN… APPROACH            !       ! Unknown bird             1  NONE   
 IL     CHICAGO O'HARE INT… CLIMB          12000        ! Unknown bird             1  PRECA… 
1› faa-wildlife-strikes|                                               h   go-left      73448 rows