Using Python to create directories and move files

Today I needed to move 435 TIFF images from the San Marcos Daily Record Negative Collection into folders based on a section of their filename.

The filenames include the following sections: SMDR_1959-185_001.tif

  1. SMDR, the collection code for the items digitized
  2. 1959, the year the negatives are from
  3. 185, the physical folder the negatives are from
  4. 001, the index number for the image
  5. tif, the file’s extension

If there is more than 1 negative in folder 185, then the 2nd image will get index number 002, i.e. SMDR_1959-185_002.tif.

Example of TIFF filenames

Some of the TIFF images that needed to be sorted into folders

We organize the images in digital folders to match the physical folders, so every image in the SMDR collection from 1959 in physical folder 185 needs to be in a digital folder 185. The folder therefore only needs sections 1-3 of the filename, i.e. SMDR_1959-185.

I have the images and I know what my digital folder names need to be, but if I did all of this work by hand it would not only take a significant amount of time, I would probably make a few mistakes. This type of repetitive work is better handled by the computer because it will not only create the folders and move the files more quickly, it is much less likely to make mistakes (assuming I provide the proper instructions).

This was very quickly done in Python 3.6 on Windows 10 using less than 20 lines of code:

import os
import shutil

dir = "1959_6x6/"

for file in os.listdir(dir):
    # get all but the last 8 characters to remove
    # the index number and extension
    dir_name = file[-8]
    print(f'dir_name: {dir_name}')

    dir_path = dir + dir_name
    print(f'dir_path: {dir_path}')
    
    # check if directory exists or not yet
    if not os.path.exists(dir_path):
        os.makedirs(dir_path)

    if os.path.exists(dir_path):
        file_path = dir + file
        print(f'file_path: {file_path}')
        
        # move files into created directory
        shutil.move(file_path, dir_path)

This code doesn’t use best practices as I should be creating the directory path differently, but hey, it worked and I was able to create the directories, move the files, and write this blog post in significantly less time than it would have taken to do it manually.

Jeremy Moore

jeremy.moore@txstate.edu

Simple batch renamer for Windows 10 PowerShell

I’m currently sitting in front of one of our Windows 10 computers that does not have Cygwin or Git BASH installed so there’s no way for me to quickly rename TIFF files from command-line like I am used to . . . enter Windows PowerShell!

I need to rename the files according to the formula <filename_stub>_<###>.tif such that a file like SMDR_1950s-SF-37_May-17-2017_12-51-19.tif becomes SMDR_1950s-37_001.tif. In this case, the <filename_stub> is SMDR_1950s-37 and <###> is 001.

So I wrote a little Windows PowerShell script to help out.

Continue reading

Searching the Library of Congress with Python and their new JSON API, Part 2

This post builds on my last one: Searching the Library of Congress with Python and their new JSON API, which is why I’ve added Part 2 to the end of the title. Before we dive back into the Library of Congress‘s JSON API, some housekeeping items:

  1. Even though the Library of Congress’s website is loc.gov, the abbreviation for Library of Congress is LC
    1. I tried to find a press-ready image of NBC‘s old The More You Know logo I could add here, but
      1. the updated logo doesn’t make me hear the jingle in my head
      2. I did find Megan Garber‘s 2014 article covering the PSA series for The Atlantic that has some classic video I enjoyed
  2. As of October 2017, LC has expressly stated in a disclaimer that their JSON API is a work in progress. Use at your own risk!! We might (will likely) change this!

Recap on Stryker’s Negatives Project

I recently came across Michael Bennett‘s article Countering Stryker’s Punch: Algorithmically Filling the Black Hole in the latest edition of code4lib: <– GREAT STUFF!

He’s using Adobe Photoshop  and GIMP to digitally restore blank areas in images due to a hole punch having been taken to the physical negative.

Current Task

Use the Library of Congress’s JSON API to download all of the hole punch images and their associated metadata.

Continue reading

Searching the Library of Congress with Python and their new JSON API

I recently read Michael J. Bennett‘s article, Countering Stryker’s Punch: Algorithmically Filling the Black Hole, in the latest edition of code4lib and wow, great stuff!

Bennett is comparing Adobe Photoshop’s Content Aware Fill tool and a GIMP technique I haven’t seen before as part of a workflow to digitally restore areas in photographs with no picture due to a hole punch having been taken to the physical negative. Part of Roy Stryker’s legacy at the FSA. You can read more about the negatives in Bennett’s article, this recent feature by Alan Taylor in The Atlantic and in Eric Banks’s review of what sounds like an amazing gallery show by William E. Jones in NY in 2010 in The Paris Review.

Library of Congress Search Results totaled 2,474

Library of Congress Search Results

I have been dipping my toes into computer vision recently and thought expanding upon Bennett’s work seems a good and educating challenge. A quick search at the Library of Congress returns nearly 2,500 results so this would also be a pretty good-sized data set to work on for a beginner with a bunch of exceptions and tweaking for edge-cases, but also a LOT of images and metadata to gather!

Continue reading