[To Staging] Throttle registration, remove dead code, accent-insensitive search

ARCHITECTURE.md

@@ -67,7 +67,6 @@ The ROR API provides:
 | App server | Phusion Passenger + Nginx (`vendor/docker/webapp.conf`) |
 | Container | Docker (`Dockerfile` based on `phusion/passenger-python312:3.2.0`) |
 | Observability | Sentry (`sentry-sdk` 1.45.1), django-prometheus 2.4.1 |
-| Feature flags | LaunchDarkly (`rorapi/common/features.py`; `launchdarkly-server-sdk` 7.6.1) |
 | Email | django-ses 4.8.0 (client ID registration emails) |
 | External packages | `update_address` (Geonames enrichment), `jsonschema` 3.2.0, `rapidfuzz` 3.6.1, `boto3` (unpinned), `pandas` 2.2.3 |
 
@@ -107,7 +106,7 @@ ror-api/
 │   │   ├── record_template.json
 │   │   ├── ror_schema_v2_1.json  # Vendored JSON schema for write validation
 │   │   └── index_template_es7.json  # ES index template + mappings
-│   ├── management/commands/  # CLI indexing and legacy GRID tools
+│   ├── management/commands/  # CLI indexing and data setup
 │   ├── migrations/           # Django migrations (Client model)
 │   └── tests/                # Unit, integration, functional, affiliation suites
 └── vendor/docker/            # Nginx, env, Terraform var templates for deploy
@@ -333,7 +332,6 @@ Loaded from environment and optional root `.env` file (`python-dotenv`).
 | `ROUTE_USER`, `TOKEN` | Admin API authentication |
 | `ROR_BASE_URL` | Base URL configuration |
 | `SENTRY_DSN` | Error reporting |
-| `LAUNCH_DARKLY_KEY` | Feature flags |
 | `SINGLE_SEARCH_DEFAULT` | Default affiliation matcher (`True`/`False`) |
 | `ENABLE_BEHAVIORAL_LIMITING` | Rate limiting toggle (edge behavior) |
 | `SECRET_KEY` | Django secret (falls back to a hardcoded default if unset; `DEBUG` is always `False`) |
@@ -400,9 +398,7 @@ Deploy mechanism: GitHub Action updates `_ror-api-*.auto.tfvars` in the `new-dep
 
 ## Legacy Code
 
-Commands prefixed with `legacy*` (GRID conversion, old upgrade paths) are **non-functional** — referenced data was moved to ror-data. GRID-based generation ended March 2022. Do not extend or rely on these unless explicitly reviving historical tooling.
-
-`settings.py` still contains commented GRID/ROR_DUMP version history for reference. `GRID_REMOVED_IDS` is an empty list retained for a check in `retrieve_organization`.
+GRID-based generation ended March 2022. The `legacy*` management commands, the LaunchDarkly call site, and the empty `GRID_REMOVED_IDS` check have been removed. `settings.py` still contains a short comment noting that ROR is no longer based on GRID. Historical GRID/ROR dump files live in [ror-data](https://github.com/ror-community/ror-data).
 
 ---
 

README.md

@@ -107,49 +107,9 @@ The API uses the v2 schema only. Use `-s 2` when indexing a data dump. A v2 form
         python manage.py setup v1.32-2023-09-14-ror-data -s 2 -t
 
 
-## LEGACY: Converting GRID data to ROR  (process used prior to Mar 2022)
+## GRID history (prior to Mar 2022)
 
-Steps used prior to Mar 2022:
-- Convert latest GRID dataset to ROR (including assigning ROR IDs)
-- Generate ROR data dump
-- Index ROR data dump into Elastic Search
-
-As of Mar 2022 ROR is no longer based on GRID. Record additions/updates and data deployment is now managed in https://github.com/ror-community/ror-records using the ```indexror``` command described above.
-
-Steps below no longer work, as data files have been moved to [ror-data](https://github.com/ror-community/ror-data). This information is being maintained for historical purposes.
-
-Management commands used in this process no longer work and are pre-pended with "legacy".
-
-
-To import GRID data, you need a system where `setup` has been run successfully. Then first update the `GRID` variable in `settings.py`, e.g.
-
-```
-GRID = {
-    'VERSION': '2020-03-15',
-    'URL': 'https://digitalscience.figshare.com/ndownloader/files/22091379'
-}
-```
-
-And, also in `settings.py`, set the `ROR_DUMP` variable, e.g.
-
-```
-ROR_DUMP = {'VERSION': '2020-04-02'}
-```
-
-Then run this command: `./manage.py upgrade`.
-
-You should see this in the console:
-
-```
-Downloading GRID version 2020-03-15
-Converting GRID dataset to ROR schema
-ROR dataset created
-ROR dataset ZIP archive created
-```
-
-This will create a new `data/ror-2020-03-15` folder, containing a `ror.json` and `ror.zip`. To finish the process, add the new folder to git and push to the GitHub repo.
-
-To install the updated ROR data, run `./manage.py setup`.
+Before March 2022, ROR records were derived from GRID: convert the GRID dataset, generate a dump, and index it. That pipeline and its management commands have been removed. Record additions/updates and data deployment are now managed in https://github.com/ror-community/ror-records using the `indexror` command described above. Historical GRID/ROR dump files live in [ror-data](https://github.com/ror-community/ror-data).
 
 ## Create new record file (v2 only)
 

rorapi/common/queries.py

@@ -1,5 +1,6 @@
 import re
 import json
+import unicodedata
 from titlecase import titlecase
 from collections import defaultdict
 
@@ -11,7 +12,7 @@
     Organization as OrganizationV2,
     ListResult as ListResultV2
 )
-from rorapi.settings import GRID_REMOVED_IDS, ROR_API, ES_VARS
+from rorapi.settings import ROR_API, ES_VARS
 from rorapi.common.es_utils import ESQueryBuilder
 
 from urllib.parse import unquote
@@ -187,6 +188,14 @@ def validate(params):
     return Errors(errors) if errors else None
 
 
+def nfc(value):
+    """Normalizes a string to Unicode NFC so decomposed (NFD) input matches
+    the precomposed characters indexed in Elasticsearch"""
+    if isinstance(value, str):
+        return unicodedata.normalize("NFC", value)
+    return value
+
+
 def build_search_query(params):
     """Builds search query from API parameters"""
 
@@ -198,20 +207,21 @@ def build_search_query(params):
             del params["all_status"]
 
     if "query.advanced" in params:
-        qb.add_string_query_advanced(params.get("query.advanced"))
+        qb.add_string_query_advanced(nfc(params.get("query.advanced")))
     elif "query" in params:
-        ror_id = get_ror_id(params.get("query"))
+        query = nfc(params.get("query"))
+        ror_id = get_ror_id(query)
         if ror_id is not None:
             qb.add_id_query(ror_id)
         else:
-            qb.add_string_query(params.get("query"))
+            qb.add_string_query(query)
     else:
         qb.add_match_all_query()
 
     if "filter" in params or (not "all_status" in params):
         filters = [
             f.split(":")
-            for f in filter_string_to_list(params.get("filter", ""))
+            for f in filter_string_to_list(nfc(params.get("filter", "")))
             if f
         ]
         # normalize filter values based on casing conventions used in ROR records
@@ -247,10 +257,10 @@ def build_search_query(params):
 
     qb.add_aggregations(
         [
-            ("types", "types"),
-            ("countries", "locations.geonames_details.country_code"),
-            ("continents", "locations.geonames_details.continent_code"),
-            ("statuses", "status"),
+            ("types", "types.raw"),
+            ("countries", "locations.geonames_details.country_code.raw"),
+            ("continents", "locations.geonames_details.continent_code.raw"),
+            ("statuses", "status.raw"),
         ]
     )
 
@@ -280,19 +290,6 @@ def search_organizations(params):
 
 def retrieve_organization(ror_id):
     """Retrieves the organization of the given ROR ID"""
-    if any(ror_id in ror_id_url for ror_id_url in GRID_REMOVED_IDS):
-        return (
-            Errors(
-                [
-                    "ROR ID '{}' was removed by GRID during the time period (Jan 2019-Mar 2022) "
-                    "that ROR was synced with GRID. We are currently working with the ROR Curation Advisory Board "
-                    "to restore these records and expect to complete this work in 2022".format(
-                        ror_id
-                    )
-                ]
-            ),
-            None,
-        )
     search = build_retrieve_query(ror_id)
     results = search.execute()
     total = results.hits.total.value

rorapi/common/views.py

@@ -5,6 +5,7 @@
 from django.views import View
 from django.shortcuts import redirect
 from rest_framework.permissions import BasePermission
+from rest_framework.throttling import AnonRateThrottle
 from rest_framework.views import APIView
 from rest_framework.parsers import FormParser, MultiPartParser
 from rorapi.settings import DATA
@@ -43,7 +44,16 @@
 from rorapi.v2.models import Client
 from rorapi.v2.serializers import ClientSerializer
 
+
+class ClientRegistrationThrottle(AnonRateThrottle):
+    """Tight anonymous limit for client-ID registration (settings.py left unchanged)."""
+
+    rate = "5/hour"
+
+
 class ClientRegistrationView(APIView):
+    throttle_classes = [ClientRegistrationThrottle]
+
     def post(self, request, version='v2'):
         serializer = ClientSerializer(data=request.data)
         if serializer.is_valid():