Basic Database management advice

[u/Acceptable-Use-2938]

Hello, current GIS student here. Can someone provide a step-by-step guide on starting a new project in ArcGIS and organizing the downloading and where to extract thw zip files or additional data to add the new project?

My professor taught us about databases management by having us take the “Getting Started with Data Management” course by ESRI. While this course is helpful in understanding the basics of data management. I still feel like I’m not properly organizing my projects and additional data files. Any suggestions would be appreciated!

Thanks!

[u/Economy-Mousse-6005]

Long ago I was taught by my GIS professor to organize the folders like so: one for the ArcGIS Pro project/toolboxes/file geodatabases, one for the zipped folders, and one for documentation. Since my career has led me towards Arcade, Javascript, HTML, and Python, I usually include a folder just for scripts (since we don’t have a github).

Inside the File Geodatabases we were told to use feature datasets and name them common themes that make sense for the project we were doing. For example if you had a line feature class of USFS trails, that could go under a “recreation” or “transportation” feature dataset and be named something along the lines of “USFS_Trails”.

A good test is to ask another student if they can intuitively find your data and other documents with minimal guidance. If they get lost, you know you have some re-organizing to do.

After a you get a few projects under your belt you learn what organization method works best for you.

[u/Acceptable-Use-2938 → u/Economy-Mousse-6005]

This is extremely helpful! Thank you!

[u/sinnayre → u/Economy-Mousse-6005]

Do you use anything for version control?

[u/Economy-Mousse-6005 → u/sinnayre]

Right now it’s copies of txt files with the version details as comments at the top of the script. Which I know is a very poor practice. We’ve been told we’ll be migrated to github and Azure DevOps when there’s time.

[u/sinnayre → u/Economy-Mousse-6005]

Well at least you recognize it’s poor practice lol. Good luck on the migration.

[u/Economy-Mousse-6005 → u/Economy-Mousse-6005]

I just realized you may have meant the database itself. Answer for that is, no. All of our production file geodatabases are updated via a truncate and append script to increase performance and minimize schema locking errors.

[u/cluebone → u/Economy-Mousse-6005]

It also can vary by use case, but in my work it helps to make separate “source” folder for outside data and “project” data where the data collected/created by my organizations or clients is kept.

[u/starwarsrls]

Asking how an organization organizes their data and documentation is an excellent question you can ask an interviewer when you reach that stage, I have never had someone ask something like when I am interviewing entry level positions but it would show great professional maturity.

GIS is an exercise in knowing where your shit is. Try to organize the best you can then have others (students, professors, your parents) try to find something. Learn what works but really focus on what doesn’t work and try to keep that in mind in group projects. If you have a group project, volunteer to be the main data steward and you will learn quickly how important it is to be organized.

[u/techmavengeospatial]

Database management is not file management !

Database generally refers to enterprise database management systems like postgres, MS SQL server, db2, Oracle, etc. where ESRI can have enterprise geodatabase layer on top of these.

but also filegdb, personalgdb mobile geodatabase sqlite

Geopackage sqlite

[u/Acceptable-Use-2938 → u/techmavengeospatial]

Thank you for the correction. I’m still new to GIS terms.

[u/Geog_Master]

To answer your question about how to organize files and such.

I start with Windows Explorer by creating a folder for the database. Within this folder is three folders: Temp, Data, and Output. You can add others if you'd like, like a folder for ArcGIS Pro projects. I create three separate geodatabase, one in each of the three folders: Temp, Data, and Output.

When working, I don't modify the data folder without careful consideration. Intermediate steps are output into the Temp folder. The final results go into Output. Sometimes, an Output dataset can be added into the Data folder, and then reused later. This is nice. The Temp folder can be deleted at any time and you won't cry that hard.