Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- {
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Examples of useful file utilities"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## The os library for listing files and making directories"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Say I have a directory on my computer filled with *.ab1* files that I need to work with."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "metadata": {},
- "outputs": [],
- "source": [
- "abi_dir = '/Users/triznam/Downloads/rev'"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "You can then use os.listdir to get a list of all of the files in that directory."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "metadata": {},
- "outputs": [],
- "source": [
- "for file in os.listdir(abi_dir):\n",
- " print(file)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Notice Macs includes this weird \".DS_Store\" file in every folder, so you should build in some sort of filter."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "metadata": {},
- "outputs": [],
- "source": [
- "for file in os.listdir(abi_dir):\n",
- " if file.endswith('.ab1'):\n",
- " print(file)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Now, say for example, you want to move all of these files to another directory. First, let's make a new directory with *os.makedirs*. This command is similar to `mkdir -p` in Unix, where is automatically creates parent directories if needed, so I used this instead of *os.mkdir*.\n",
- "\n",
- "I also include a check to see if the directory already exists, since it will error out if it has already been made. I honestly didn't realize this until I got through writing this up, and then tried Restart and Run All on the notebook."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 5,
- "metadata": {},
- "outputs": [],
- "source": [
- "good_abis_path = '/Users/triznam/Documents/ab1_files/good_ones/'\n",
- "if not os.path.exists(good_abis_path):\n",
- " os.makedirs(good_abis_path)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Copying files with shutil"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "For some reason, the command to copy files is in a different library than os 🤷♂️, so we'll have to import the *shutil* library."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 6,
- "metadata": {},
- "outputs": [],
- "source": [
- "import shutil"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Now let's run through the same for-loop as before to copy each file to the new directory. The *shutil.copyfile* command requires the full path, so we'll also need to use *os.path.join* to combine the directory with the file name."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 7,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Current filepath: /Users/triznam/Downloads/rev/647767_HCO2198_E01.ab1\n",
- "New filepath: /Users/triznam/Documents/ab1_files/good_ones/647767_HCO2198_E01.ab1\n",
- "--------\n",
- "Current filepath: /Users/triznam/Downloads/rev/647766_HCO2198_C01.ab1\n",
- "New filepath: /Users/triznam/Documents/ab1_files/good_ones/647766_HCO2198_C01.ab1\n",
- "--------\n",
- "Current filepath: /Users/triznam/Downloads/rev/647768_HCO2198_F01.ab1\n",
- "New filepath: /Users/triznam/Documents/ab1_files/good_ones/647768_HCO2198_F01.ab1\n",
- "--------\n",
- "Current filepath: /Users/triznam/Downloads/rev/647769_HCO2198_G01.ab1\n",
- "New filepath: /Users/triznam/Documents/ab1_files/good_ones/647769_HCO2198_G01.ab1\n",
- "--------\n",
- "Current filepath: /Users/triznam/Downloads/rev/647764_HCO2198_A01.ab1\n",
- "New filepath: /Users/triznam/Documents/ab1_files/good_ones/647764_HCO2198_A01.ab1\n",
- "--------\n",
- "Current filepath: /Users/triznam/Downloads/rev/647762_HCO2198_D01.ab1\n",
- "New filepath: /Users/triznam/Documents/ab1_files/good_ones/647762_HCO2198_D01.ab1\n",
- "--------\n",
- "Current filepath: /Users/triznam/Downloads/rev/647765_HCO2198_B01.ab1\n",
- "New filepath: /Users/triznam/Documents/ab1_files/good_ones/647765_HCO2198_B01.ab1\n",
- "--------\n",
- "Current filepath: /Users/triznam/Downloads/rev/647763_HCO2198_B07.ab1\n",
- "New filepath: /Users/triznam/Documents/ab1_files/good_ones/647763_HCO2198_B07.ab1\n",
- "--------\n",
- "Current filepath: /Users/triznam/Downloads/rev/647770_HCO2198_H01.ab1\n",
- "New filepath: /Users/triznam/Documents/ab1_files/good_ones/647770_HCO2198_H01.ab1\n",
- "--------\n"
- ]
- }
- ],
- "source": [
- "for file in os.listdir(abi_dir):\n",
- " if file.endswith('.ab1'):\n",
- " current_file_path = os.path.join(abi_dir, file)\n",
- " print('Current filepath: {}'.format(current_file_path))\n",
- " new_file_path = os.path.join(good_abis_path, file)\n",
- " print('New filepath: {}'.format(new_file_path))\n",
- " shutil.copyfile(current_file_path, new_file_path)\n",
- " print('-'*8)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Let's do a `ls` command to manually check that the files are now in the new directory."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 8,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "647762_HCO2198_D01.ab1 647765_HCO2198_B01.ab1 647768_HCO2198_F01.ab1\r\n",
- "647763_HCO2198_B07.ab1 647766_HCO2198_C01.ab1 647769_HCO2198_G01.ab1\r\n",
- "647764_HCO2198_A01.ab1 647767_HCO2198_E01.ab1 647770_HCO2198_H01.ab1\r\n"
- ]
- }
- ],
- "source": [
- "!ls ~/Documents/ab1_files/good_ones"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Zipping a directory with shutil"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "There is a *zipfile* library in the Python standard library, but I just learned while Googling that there's an easier version right in shutil which is more straightforward, so let's use that."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Say I wanted to zip up the \"good\" ab1 files and put it on the Desktop, I would use the following command:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 10,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "'/Users/triznam/Desktop/good_ones.zip'"
- ]
- },
- "execution_count": 10,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "shutil.make_archive('/Users/triznam/Desktop/good_ones', #The filepath of the zipfile (minus .zip)\n",
- " 'zip', #What we want to zip it with -- also can use \n",
- " good_abis_path)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.7.1"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 2
- }
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement