The OpenMindat Python package is designed to facilitate querying and retrieving data on minerals and geomaterials from the Mindat API. It provides classes for detailed queries based on various attributes like IMA status, keywords, and specific geomaterial properties.
GitHub Repository: OpenMindat Python Package
- Get Started
- Endpoint Descriptions
- Use Cases
- 0. Setup
- 1. Perform Detailed Queries on Geomaterials
- 2. Retrieve IMA-Approved Minerals
- 3. Search Geomaterials Using Keywords
- 4. Retrieve Localities
- 5. Retrieve Type Localities for IMA-Approved Mineral Species
- 6. Retrieve Locality Occurrences for Single Mineral Species
- 7. Download Mineral Species with Their Locality Records
- 8. Retrieve Locality Name Translations
- 9. List the Values of a Geomaterial Field
- Errors and Large Queries
- Documentation and Relevant Links
- Contact Us
- License
- Authors
- Citations
- Acknowledgments
- Change Logs
pip install openmindatOpenMindat requires Python 3.10 or later.
import openmindat
| Endpoint | Classes | Description |
|---|---|---|
| Dana8 | - DanaRetriever() | Search query to return information about the Dana-8 classification standard. |
| Geomaterials | - GeomaterialRetriever() - GeomaterialIdRetriever() - GeomaterialDictRetriever() |
Search query to return information about mindat database items such as id, name, group id etc. |
| Geomaterial_search | - GeomaterialSearchRetriever() | Query to search for mindat database entries based on search keywords. |
| Localities | - LocalitiesRetriever() - LocalitiesIdRetriever() |
Search query to return information about different localities, for examples the main elements present in the Jegdalek ruby deposit in Afghanistan |
| Localities_Age | - LocalitiesAgeRetriever() - LocalitiesAgeIdRetriever() |
Search query to return locality age details. |
| Localities_Status | - LocalitiesStatusRetriever() - LocalitiesStatusIdRetriever() |
Search query to return information about the status type of localities. For example, abandoned is a status type. |
| Localities_Type | - LocalitiesTypeRetriever() - LocalitiesTypeIdRetriever() |
Search query to return information about the type of localities. For example, a Mining Field is a locality type. |
| Locality_Translations | - LocalityTranslationsRetriever() - LocalityTranslationsIdRetriever() |
Locality names in other languages, e.g. the French name of a locality. |
| Minerals-IMA | - MineralsIMARetriever() - MineralsIdRetriever() |
Search query to return IMA details for a mineral. For example the year it's IMA status was approved. |
| Nickel_Strunz | - StrunzRetriever() | Search query to return information about the Nickel-Strunz-10 classification standard. |
DanaRetriever and StrunzRetriever need a sub-listing or an id, e.g. DanaRetriever().groups() or StrunzRetriever().classes(). The Mindat API has no plain list for these endpoints.
The Mindat API no longer provides the countries, locgeoregion2, locobject and photocount endpoints. For old code, CountriesListRetriever, CountriesIdRetriever, GeoRegionRetriever, LocobjectRetriever and PhotoCountRetriever can still be imported from their modules (e.g. openmindat._countries), but they raise EndpointUnavailableError when queried. They will be removed in version 0.2.0.
If you do not have a Mindat API key yet, see How to Get My Mindat API Key or Token?. Then use one of these options:
# Option 1: environment variable (good for scripts and CI)
import os
os.environ["MINDAT_API_KEY"] = "Your_Mindat_API_Key"
# Option 2: save the key once for later sessions
import openmindat
openmindat.set_api_key("Your_Mindat_API_Key", persist=True)Option 3: just run a query. If no key is found, OpenMindat asks for one and saves it.
A saved key is stored in ~/.config/openmindat/credentials.yaml (%APPDATA%\openmindat\credentials.yaml on Windows), readable only by you. If you used OpenMindat 0.1.3 or earlier, the .apikey.yaml file in your working folder is still read, and it is copied to the new location automatically.
To check which key is used and whether Mindat accepts it:
import openmindat
print(openmindat.api_key_status())
# Mindat API key abcd…wxyz from env: validfrom openmindat import GeomaterialRetriever
gr = GeomaterialRetriever()
# Print out the available functions for a class
gr.available_methods()from openmindat import GeomaterialRetriever
gr = GeomaterialRetriever()
# Typo check
gr.elements_in('Cu')
'''>>> AttributeError: 'GeomaterialRetriever' object has no attribute 'elements_in',
Available methods: ['_init_params', 'available_methods', 'bi_max', 'bi_min', 'cleavagetype', 'color',
'colour', 'crystal_system', 'density_max', 'density_min', 'diaphaneity', 'diapheny', 'el_essential',
'el_exc', 'el_inc', 'elements_exc', 'elements_inc', 'entrytype', 'expand', 'fields', 'fracturetype',
'get_dict', 'groupid', 'hardness_max', 'hardness_min', 'id_in', 'id_max', 'id_min', 'ima', 'ima_notes',
'ima_status', 'lustretype', 'meteoritical_code', 'meteoritical_code_exists', 'name', 'non_utf', 'omit',
'optical2v_max', 'optical2v_min', 'opticalsign', 'opticaltype', 'ordering', 'page', 'page_size',
'polytypeof', 'q', 'ri_max', 'ri_min', 'save', 'saveto', 'streak', 'synid', 'tenacity', 'updated_at',
'varietyof', 'verbose']. Did you mean: 'elements_inc'?'''from openmindat import GeomaterialRetriever
gr = GeomaterialRetriever()
gr.density_min(2.0).density_max(5.0).crystal_system("Hexagonal")
gr.elements_exc("Au,Ag")
gr.save()from openmindat import MineralsIMARetriever
mir = MineralsIMARetriever()
mir.saveto("./mindat_data", 'my_filename')
# Filters work as on GeomaterialRetriever, e.g. hexagonal minerals with hardness >= 8:
mir = MineralsIMARetriever()
mir.crystal_system("Hexagonal").hardness_min(8)
print(mir.get_dict())
MineralsIMARetriever().fields(...)currently fails when the field list includesname. This is a Mindat server issue that has been reported.
from openmindat import GeomaterialSearchRetriever
gsr = GeomaterialSearchRetriever()
gsr.geomaterials_search("quartz, green, hexagonal")
gsr.save("filename")
# Alternatively, you can get the list object directly:
gsr = GeomaterialSearchRetriever()
gsr.geomaterials_search("ruby, red, hexagonal")
print(gsr.get_dict())
# Only the exact name match:
gsr = GeomaterialSearchRetriever()
gsr.geomaterials_search("quartz").e(1)
print(gsr.get_dict())from openmindat import LocalitiesRetriever
# Download Localities for certain state
lr = LocalitiesRetriever()
lr.country("USA").txt("Idaho")
lr.save()
# Alternatively, you can get the list object directly:
lr = LocalitiesRetriever()
lr.country("Canada").description("mine")
print(lr.get_dict())
# Localities in Norway at the top two hierarchy levels with at least 10 sub-localities,
# including the ids of their minerals:
lr = LocalitiesRetriever()
lr.country("Norway").level_lte(2).sublocs_gte(10).expand("geomaterials")
print(lr.get_dict())from openmindat import GeomaterialRetriever
gr = GeomaterialRetriever()
gr.ima(True).expand("type_localities")
gr.saveto("./mindat_data")Please consider using only one mineral species ID for querying localities occurrences since this query might result in many records and exceed the server limitation.
from openmindat import GeomaterialRetriever
gr = GeomaterialRetriever()
gr.expand("locality").id_in("3337") # 3337 is Quartz
gr.saveto("./mindat_data")expand("locality") adds the ids of every locality where a mineral is recorded on Mindat. It is the usual starting point for a mineral occurrence dataset. Include locality in fields; otherwise it is dropped from the output.
from openmindat import GeomaterialRetriever, LocalitiesRetriever
# The first 5 IMA-approved minerals with their locality ids
gr = GeomaterialRetriever()
gr.ima(True).expand("locality").fields("id,name,ima_formula,locality")
gr.page_size(5).page(1)
minerals = gr.get_dict()["results"]
for m in minerals:
print(f"{m['name']:<20} {len(m['locality']):>5} localities")To turn the locality ids into names and coordinates, look them up with LocalitiesRetriever().id_in(...). Query in batches of about 200 ids: common minerals have thousands of localities, which is too long for a single request.
mineral = minerals[2] # Abernathyite
ids = mineral["locality"]
localities = []
for start in range(0, len(ids), 200):
batch = ",".join(map(str, ids[start:start + 200]))
lr = LocalitiesRetriever().id_in(batch).fields("id,txt,latitude,longitude")
localities += lr.get_dict()["results"]
for loc in localities:
print(loc["id"], loc["latitude"], loc["longitude"], loc["txt"])Mindat stores "no coordinates" as latitude = longitude = 0.0 (typically for regions such as countries or districts), not as empty values.
To download all IMA-approved minerals with their locality ids, remove .page(1) and save to a file. This covers about 6,200 minerals and over one million mineral–locality pairs (about 25 MB), so it takes a while:
gr = GeomaterialRetriever()
gr.ima(True).expand("locality").fields("id,name,ima_formula,locality").page_size(50)
gr.saveto("./mindat_data", "minerals_with_localities")from openmindat import LocalityTranslationsRetriever
# French names of locality 14090 (Belgium)
ltr = LocalityTranslationsRetriever()
ltr.lt_loc(14090).lt_iso("fr")
print(ltr.get_dict())from openmindat import GeomaterialDictRetriever
# All crystal systems used by Mindat. field() is required.
gdr = GeomaterialDictRetriever()
print(gdr.field("csystem").get_dict())When the Mindat API cannot answer a query, OpenMindat raises an exception. Each one is also a ValueError, so existing except ValueError: code keeps working.
| Exception | Raised when |
|---|---|
MindatRequestError |
The Mindat API rejected the request (HTTP 4xx), e.g. an unknown id or a missing parameter. The message quotes the server's reason. |
MindatServerError |
The Mindat server failed: an HTTP 5xx error, a gateway timeout, a rate limit or a network problem. |
EndpointUnavailableError |
The class wraps an endpoint the Mindat API no longer provides. |
from openmindat import GeomaterialRetriever, MindatServerError
gr = GeomaterialRetriever().ima(True).crystal_system("Monoclinic").expand("locality")
try:
data = gr.get_dict()
except MindatServerError as error:
print(error)
data = {"results": error.partial_results} # records downloaded before the failureTo get error responses back as data instead, as in version 0.1.3 and earlier, set openmindat.MindatApi.RAISE_HTTP_ERRORS = False.
Large queries are downloaded in pages of 1500 records. If the server times out on a page, OpenMindat requests that page again in smaller pieces, and it waits when the server asks it to slow down. To make big downloads faster, request only the fields you need. For example, GeomaterialRetriever().fields("id,name,mindat_formula") made pages about 25 times smaller in our tests.
-
Built-in Help:
To explore detailed class and method documentation within the OpenMindat package, use Python's built-in help() function. This provides direct access to docstrings, showcasing usage examples and parameter details. Example:
from openmindat import GeomaterialRetriever
help(GeomaterialRetriever)The help() is also available for the specific functions:
from openmindat import MineralsIMARetriever
help(MineralsIMARetriever.fields)Press q to exit the help interface.
For further assistance or feedback, feel free to contact the development team at jiyinz@uidaho.edu.
Project Licence: Apache
Mindat Data License: CC BY-NC-SA 4.0 DEED
The Mindat API is currently in beta test, and while access is free for all, please note that the data provided are not yet licensed for redistribution and are for private, non-commercial use only. Once launched, data will be available under an open-access license, but please always check the terms of use of the license before reusing these data.
Jiyin Zhang, Cory Clairmont, Xiaogang Ma
If you use the data or code from this repository, please cite it as indicated below.
@misc{OpenMindat,
author = {Jiyin Zhang and Cory Clairmont and Xiaogang Ma},
title = {OpenMindat: A Python Package for Geomaterial Data Analysis and Retrieval from Mindat API},
year = {2024},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/ChuBL/OpenMindat}},
note = {Version 0.1.4}
}
Additionally, you should also reference the following paper:
Ma, X., Ralph, J., Zhang, J., Que, X., Prabhu, A., Morrison, S.M., Hazen, R.M., Wyborn, L. and Lehnert, K., 2024. OpenMindat: Open and FAIR mineralogy data from the Mindat database. Geoscience Data Journal, 11(1), pp.94-104. https://doi.org/10.1002/gdj3.204.
- This work is supported by NSF, Award #2126315.