# I Removed One Line of Django Code and My API Got 95ms Faster ## The Problem Tripvento's city-matrix endpoint returns every hotel in a city scored against 14 traveler personas. At 33 cities it felt fast. At 212 cities, with 24,000+ hotels, cold responses were creeping toward 700ms and the payload for a single city had quietly reached **1.8MB**. Nothing was broken. No alerts were firing. But I knew what that feeling meant. I was about to hit a wall. Before touching a single line of code, I opened a Django shell and started measuring. * * * ## Profile First. Fix Second. The biggest mistake I see in performance debugging is jumping straight to solutions. Add an index. Throw in a cache. Upgrade the droplet. All of these can mask the real problem while adding complexity. I isolated each layer independently: ```python import time from hotels.models import StagingHotel, StagingHotelIntent from hotels.serializers import HotelIntentMatrixSerializer from django.db.models import Prefetch # layer 1 raw query start = time.time() qs = list(StagingHotel.objects.filter( destination__name__iexact='Savannah', permanently_closed=False ).select_related( 'destination', 'destination__region', 'neighborhood_obj' ).prefetch_related( 'metrics', Prefetch('intents', queryset=StagingHotelIntent.objects.filter(is_eligible=True), to_attr='prefetched_intents') )[:15]) print(f'Query: {(time.time()-start)*1000:.0f}ms') # layer 2 serializer start = time.time() data = HotelIntentMatrixSerializer(qs, many=True, context={'demo_mode': True}).data print(f'Serialize: {(time.time()-start)*1000:.0f}ms') # layer 3 JSON start = time.time() json_str = json.dumps(data) print(f'JSON dump: {(time.time()-start)*1000:.0f}ms, {len(json_str)//1024}KB') ``` Results: ```plaintext City Matrix: 176ms Serialize: 19ms JSON dump: 8ms ``` That's 203ms in Python. But cold curl requests were hitting 679ms. The missing ~470ms was Django middleware, DRF request parsing, and network. Knowing that told me where *not* to look. The Python layer was the fixable part. I had 203ms to work with. * * * ## Fix 1: The 95ms Line The single biggest win came from one method call I hadn't thought twice about. In `CityMatrixViewSet.list()`, I was calling `queryset.count()` before slicing — to populate a `total_available` field in the response. Reasonable sounding. Completely wasteful. ```python # This ran on every single request total_count = queryset.count() # Full table scan - 95ms ``` For demo tier users capped at 15 results, I didn't need the exact count. I just needed to know if there were more. ```python # Fetch 16, check if a 16th exists limited_qs = list(queryset[:result_limit + 1]) has_more = len(limited_qs) > result_limit queryset = limited_qs[:result_limit] # total_count gone entirely ``` 95ms eliminated. One change. No tradeoff. The broader lesson: `count()` in Django triggers a `SELECT COUNT(*)` where Postgres must traverse the index and verify row visibility for every matching row due to MVCC. If you're calling it on a large filtered queryset just to display a number, ask whether that number is actually used. * * * ## Fix 2: The Prefetch Cache That Wasn't Django's `prefetch_related` is supposed to batch related object lookups. You set it up in `get_queryset()`, and when the serializer accesses related objects, it reads from the in-memory cache instead of hitting the database again. Except I wasn't actually using it. In my serializer: ```python def _get_latest_metric(self, obj): if not hasattr(obj, '_latest_metric_cache'): # DB query per hotel obj._latest_metric_cache = obj.metrics.first() return obj._latest_metric_cache ``` `.first()` on a related manager bypasses the prefetch cache entirely. Django evaluates it as a fresh queryset. With 15 hotels per response, that's 15 extra queries silently, on every request. The fix is reading from the prefetch cache directly: ```python def _get_latest_metric(self, obj): if not hasattr(obj, '_latest_metric_cache'): prefetched = getattr(obj, '_prefetched_objects_cache', {}).get('metrics') if prefetched is not None: obj._latest_metric_cache = prefetched[0] if prefetched else None else: obj._latest_metric_cache = obj.metrics.first() return obj._latest_metric_cache ``` Same result. Zero extra queries. > Note: If you used `to_attr='prefetched_metrics'` in your Prefetch object, you can skip the internal cache dict entirely and use `getattr(obj, 'prefetched_metrics', None)` instead . * * * ## Fix 3: Index Hygiene I had five indexes on `StagingHotel`. One of them was doing nothing. ```python indexes = [ models.Index(fields=['destination']), # redundant models.Index(fields=['destination', 'permanently_closed']), # covers the above models.Index(fields=['latitude', 'longitude']), models.Index(fields=['provider_id']), models.Index(fields=['is_ready_for_production']), ] ``` A composite index on `(destination, permanently_closed)` already handles any query filtering on `destination` alone — PostgreSQL uses the leftmost columns of a composite index. The standalone `destination` index was dead weight: taking up space, slowing down writes, and contributing nothing to reads. Dropped it. The broader habit: audit your indexes the same way you audit your code. Redundant indexes aren't free — they cost write performance and memory. * * * ## Fix 4: PostgreSQL Wasn't Tuned At All My `docker-compose.yml` had no PostgreSQL configuration. Out of the box, Postgres ships with conservative defaults designed for shared hosting environments circa 2005. ```yaml command: > postgres -c shared_buffers=256MB -c effective_cache_size=768MB -c work_mem=16MB -c maintenance_work_mem=128MB -c random_page_cost=1.1 ``` `shared_buffers` tells Postgres how much RAM to use for caching data pages. The default is 128MB — laughably low for a modern server. `random_page_cost=1.1` tells the query planner that random disk reads are almost as cheap as sequential ones, which is true on SSDs and pushes it toward index scans over sequential scans. None of this requires code changes. It's configuration. It costs nothing and the gains are immediate. * * * ## Fix 5: Stop Sending Data Nobody Asked For After fixing the query layer, I looked at what I was actually sending over the wire. City-matrix at 111 hotels: **1.8MB**. That's ~16KB per hotel which included story fields, full amenity lists, nearby POI breakdowns, images, algorithm metadata duplicated on every single hotel object. The algorithm metadata was the easiest fix. I was serializing the same engine config — version, fusion strategy, radius — on every hotel in the response. 111 copies of the same object. But the structural fix was bigger: I added a `?thin=true` mode that strips hotel detail fields entirely and returns only what's needed for rendering a ranked list like `id`, `name`, `location`, `detail_url`, and all scores. ```bash # Fat — full hotel details GET /rankings/?destination=miami_fl&intent=romantic # ~466KB # Thin — scores + identifiers only GET /rankings/?destination=miami_fl&intent=romantic&thin=true # ~88KB ``` **466KB → 88KB. 81% reduction.** For city-matrix, which powers comparison UIs and pre caching thin is now the default. If a user selects a specific hotel, the frontend fetches the full detail via `detail_url`. You pay for the data when you need it, not on every list render. The previous [PostGIS article](https://blog.tripvento.com) covers how I eliminated the 55M row `HotelPOI` table from the database entirely. That's the same principle applied to storage, don't persist what you don't query. * * * ## Fix 6: The Cache Was a Time Bomb While auditing the query layer, I checked Redis and found this: ```plaintext Maxmemory Policy: noeviction Maxmemory: 25.00 MB Used: 7.21 MB RSS: 23.67 MB ``` `noeviction` is Redis's default policy. It means when memory fills up, Redis stops accepting writes and returns errors. Not "evict old keys." Not "evict least recently used." Just errors. My RSS was already at 23.67MB on a 25MB plan. I was one traffic spike away from Redis silently breaking and every cache miss becoming a cold Postgres hit. One command fixed it: ```bash heroku redis:maxmemory -a your-app --policy allkeys-lru # we've since migrated to DigitalOcean Valkey the policy is set the same way via redis-cli or your provider's dashboard. ``` `allkeys-lru` evicts the least recently used keys when memory fills up. For a cache, this is always the right policy because you'd rather lose stale data than have writes fail. The cache degrades gracefully under pressure instead of exploding. This assumes Redis is a dedicated cache. If you're also using it for Celery queues or persistent data, use `volatile-lru` instead. This isn't a performance optimization. It's a correctness fix. The "optimization" is that your cache actually keeps working when traffic spikes instead of silently becoming a liability. If you're running Redis as a cache and haven't checked your eviction policy, go check it right now. * * * ## Fix 7: The Easiest Win of All After all the query and payload work, I realized I'd never enabled gzip compression. Django ships with `GZipMiddleware` built in. It's not enabled by default. One line: ```python MIDDLEWARE = [ 'corsheaders.middleware.CorsMiddleware', 'django.middleware.gzip.GZipMiddleware', 'django.middleware.security.SecurityMiddleware', # ... ] ``` The result on city-matrix: ```plaintext Before: 1.81MB After: 261KB ``` **7x reduction. One line of middleware.** Rankings followed the same pattern, compressed responses dropped to a fraction of their original size. Any client sending `Accept-Encoding: gzip` (every modern HTTP client does) gets the compressed version automatically. Django handles the negotiation. I'd spent weeks optimizing queries and serializers. The biggest single payload reduction came from a middleware I'd simply forgotten to turn on. Check your middleware stack right now. If `GZipMiddleware` isn't there, add it before you do anything else in this list. * * * ## The Results

Metric

Before

After

Cold E2E response

~679ms

~188ms

Warm/cached response

~500ms

12ms

count() overhead

95ms per request

0ms

City-matrix payload (uncompressed)

1.81MB

261KB (gzip)

City-matrix payload (thin + gzip)

1.81MB

~88KB

Overall payload reduction

—

~95%

Redis under pressure

Errors

Graceful eviction

Same algorithm. Same infrastructure. Same DigitalOcean droplet. * * * ## The Pattern Every fix here followed the same logic as the [PostGIS migration](https://blog.tripvento.com/scaling-200-cities-by-deleting-90-percent-of-my-database): stop paying for things you don't need. * `count()` — paying for a full table scan to return a number nobody used * `.first()` bypassing prefetch — paying for N database queries when you already had the data in memory * Redundant index — paying write overhead for an index that helped nothing * 1.8MB payload — paying for CDN egress and client parsing on data that never got rendered * `noeviction` Redis — paying with production errors when you could have paid with stale cache eviction * Missing gzip — paying to transfer 7x more bytes than necessary, every single request The instinct when something is slow is to add: more cache, more indexes, more infrastructure. Sometimes the right move is to measure first and then remove the thing that shouldn't be there. * * * This is part 3 of the Building Tripvento series. Part 1 covered [deleting 55M rows](https://blog.tripvento.com/scaling-200-cities-by-deleting-90-percent-of-my-database) to scale the database. Part 2 covered the [multi LLM self healing data pipeline](https://blog.tripvento.com/how-i-built-a-self-auditing-data-pipeline-with-multiple-llms). Next up: how I built a content factory that generates destination guides at scale. *I'm* [*Ioan Istrate*](https://ioanistrate.com/)*, founder of* [*Tripvento*](https://tripvento.com) *— a hotel ranking API that scores properties against 14 traveler personas using geospatial intelligence and semantic AI. Previously worked on ranking systems at U.S. News & World Report. If you want to nerd out about Django performance or API design, let's connect on* [*LinkedIn*](https://www.linkedin.com/in/istrateioan/)*.*