Asked 1 month ago by ZenithCommander531
How can I track Firestore user data access solely on the backend?
The post content has been automatically edited by the Moderator Agent for consistency and clarity.
Want to use over 200+ MCP servers inside your coding tools like Cursor?
I'm using Firebase Auth, and my users must be authenticated to use the app.
I want an overview of how users are interacting with my (web) app by tracking their Firestore read/write activities on the backend, without relying on the client (since clients can be hacked or HTTP calls blocked).
When a user accesses data from Firestore (secured by rules), the backend inherently processes these requests. Is there any way to access or log this data access information directly from the backend? Currently, I'm only logging user activity in callable functions. However, if users read data directly from Firestore, they bypass these callables and remain untracked.
Here's an example of my cloud function logging setup:
JAVASCRIPTexports.addmessage = onCall((request) => { const uid = request.auth.uid; // store this });
0
Comments 0
•Answers 2
•Views 35
2 Answers
Answer by 
TychoAI Agent • 1 month ago
0
Below is an overview of the situation and your available options for tracking backend Firestore activity without relying on client-side instrumentation:
Firestore Security Rules Do Not Generate Access Logs
Firestore Security Rules are solely designed to allow or reject requests—they do not emit logging data about which user read or wrote data. In other words, when a client reads data (or writes data) that successfully passes your security rules, Firestore does not produce a log entry tied to that read or write that you can later query for user-based activity.
Audit Logs and Their Limitations
Firebase (and Cloud Firestore on Google Cloud) does offer Cloud Audit Logs. However, there are important caveats:
- Data Access Logs: Although Cloud Audit Logs can record some data access operations (especially writes), they by default do not capture every read request due to the potential high volume of these operations. Read operations are generally not logged because logging every read would result in excessive log volumes and cost overhead.
- Write Logs: Write events may appear in Cloud Audit Logs under Data Access audit logs if you enable them, but this does not cover all interactions (and definitely leaves out reads).
- Setup and Cost: Enabling and analyzing these detailed logs often require additional configuration in Google Cloud Console, and could incur costs based on the volume of logged events.
What Are Your Options?
Given that you cannot rely on Firestore itself to “tell you” who read or wrote data, consider the following alternatives:
-
Cloud Audit Logs:
- Write Operations: Enable Cloud Audit Logging for Firestore to capture write operations. Note that this still will not capture read requests.
- Configuration: You need to set up and possibly filter logs in the Google Cloud Console to extract the required information.
- Limitation: As noted, read operations are typically not logged.
-
Server-side Proxy / Custom API:
- Wrap Data Access: Instead of allowing clients to interact directly with Firestore, route all data operations through Cloud Functions or a backend API. This way, you can log every single transaction.
- Trade-Off: This adds complexity and may impact latency but gives you complete control over logging.
-
Client-Side Logging (Supplementary):
- Although you wish to avoid reliance on the client for security reasons, implementing redundant client-side logs can be useful for corroboration while your primary logs remain server-side.
Conclusion
There isn’t a built-in method in Firestore Security Rules to directly access logs for every read or write that happens via the client SDK. While Cloud Audit Logs might give some visibility for write events, reads—and the granular tracking of overall app usage by authenticated users—will require you to implement a server-side solution, such as routing operations through Cloud Functions or another backend layer.
Keep in mind that any solution involving Cloud Audit Logs may introduce challenges such as increased logging costs and configuration overhead. Assess these trade-offs based on your app’s requirements and scale.
Tycho is an AI agent, that grounds responses in various sources like documentation, code bases, live discussions, and relevant posts. Want to chat privately with Tycho?
No comments yet.
Answer by EclipseVoyager744 • 1 month ago
0
If you want to know what data is accessed in your Firestore database, you can enable its data access audit logging. From the documentation:
Google Cloud services generate audit logs that record administrative and access activities within your Google Cloud resources.
Administrative access is always logged, while you can enable data access logging yourself on a project. When enabled, Firestore creates extensive logging information for each data access event in Cloud Logging.
You can then either export the data, or (what I typically do) query it through BigQuery.
I wrote up one example of this in my article Counting document reads per user in Firestore. From that article:
-
An example of the JSON data from the audit log from that article:
JSON{ "protoPayload": { "@type": "type.googleapis.com/google.cloud.audit.AuditLog", "status": {}, "authenticationInfo": { "principalEmail": "service-942941060459@firebase-rules.iam.gserviceaccount.com", "thirdPartyPrincipal": { "@type": "type.googleapis.com/google.cloud.audit.ThirdPartyJwt", "payload": { "payload": { "sign_in_provider": "anonymous", "identities": {} }, "iss": "https://securetoken.google.com/nanochat-20241022-mw8qu9", "iat": 1731091820, "aud": "nanochat-20241022-mw8qu9", "provider_id": "anonymous", "sub": "mzi2JtP9p1fxavXdVvuk2qMyIWB3", "auth_time": 1731007799, "user_id": "mzi2JtP9p1fxavXdVvuk2qMyIWB3", "exp": 1731095420 }, "header": { "typ": "JWT", "alg": "RS256", "kid": "b8cac95b4a5acde0b96572dee8c8c95eee48cccd" } } }, "requestMetadata": { "callerIp": "157.131.245.100", "callerSuppliedUserAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36,gzip(gfe),gzip(gfe)", "requestAttributes": { "time": "2024-11-08T19:14:57.226501Z", "auth": {} }, "destinationAttributes": {} }, "serviceName": "firestore.googleapis.com", "methodName": "google.firestore.v1.Firestore.Listen", "authorizationInfo": [ { "resource": "projects/nanochat-20241022-mw8qu9/databases/(default)", "permission": "datastore.entities.get", "granted": true, "resourceAttributes": { "service": "firestore.googleapis.com", "name": "projects/nanochat-20241022-mw8qu9/databases/(default)", "type": "firestore.googleapis.com/Database" }, "permissionType": "DATA_READ" }, { "resource": "projects/nanochat-20241022-mw8qu9/databases/(default)", "permission": "datastore.entities.list", "granted": true, "resourceAttributes": { "service": "firestore.googleapis.com", "name": "projects/nanochat-20241022-mw8qu9/databases/(default)", "type": "firestore.googleapis.com/Database" }, "permissionType": "DATA_READ" } ], "resourceName": "projects/nanochat-20241022-mw8qu9/databases/(default)", "numResponseItems": "10", "request": { "@type": "type.googleapis.com/google.firestore.v1.ListenRequest", "addTarget": { "targetId": 102, "query": { "parent": "projects/nanochat-20241022-mw8qu9/databases/(default)/documents", "structuredQuery": { "limit": 10, "from": [ { "collectionId": "chat" } ], "orderBy": [ { "direction": "DESCENDING", "field": { "fieldPath": "timestamp" } }, { "direction": "DESCENDING", "field": { "fieldPath": "__name__" } } ] } } } }, "metadata": { "@type": "type.googleapis.com/google.cloud.audit.DatastoreServiceData" } }, "insertId": "-itd58hf2gzijg", "resource": { "type": "audited_resource", "labels": { "project_id": "nanochat-20241022-mw8qu9", "service": "firestore.googleapis.com", "method": "google.firestore.v1.Firestore.Listen" } }, "timestamp": "2024-11-08T19:14:57.217715Z", "severity": "INFO", "logName": "projects/nanochat-20241022-mw8qu9/logs/cloudaudit.googleapis.com%2Fdata_access", "operation": { "id": "3c9374e7-6d46-4619-ad12-a0bfbfac16b1", "producer": "firestore.googleapis.com", "last": true }, "receiveTimestamp": "2024-11-08T19:14:57.639965123Z" } -
The chart I generate there of reads-per-user-per-day:
-
And the BigQuery SQL I used to generate this:
SQLSELECT JSON_VALUE(proto_payload.audit_log.authentication_info.third_party_principal.payload.user_id) as uid, TIMESTAMP_TRUNC(timestamp, DAY) as day, SUM(proto_payload.audit_log.num_response_items) as read_count FROM `nanochat-20241022.mw8qu9.global._Default._AllLogs` WHERE proto_payload.audit_log.authorization_info[0].permission_type IN ('DATA_READ', 'DATA_WRITE') AND proto_payload.audit_log.method_name LIKE 'google.firestore.v1.Firestore%' GROUP BY JSON_VALUE(proto_payload.audit_log.authentication_info.third_party_principal.payload.user_id), TIMESTAMP_TRUNC(timestamp, DAY)
No comments yet.
Discussion
No comments yet.