2025-12-05 12:31:21 [common.utils.decorators] ERROR: DefaultOnExceptionError in _extract_co2_emission_value: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/usr/src/app/common/utils/decorators.py", line 16, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/app/crawlers/spiders/autotrader_details_extractor.py", line 390, in _extract_co2_emission_value
    return data["specification"]["co2Emissions"]["co2Emission"]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not subscriptable

2025-12-05 12:31:21 [common.utils.decorators] ERROR: Payload: {"id": "202509156331399", "stockItemId": "8a42b598993e4d9801994cf5be907292", "isAuction": false, "hoursUsed": null, "serviceHistory": null, "title": "Toyota Supra 3.0 3dr", "excludePreviousOwners": true, "advertisedLocations": ["at_cars", "at_profile", "at_profile_cars", "at_stock_cars", "trade_advertiser_stock"], "dueAtSeller": null, "motExpiry": null, "motInsurance": null, "lastServiceOdometerReadingMiles": null, "lastServiceDate": null, "warrantyMonthsOnPurchase": null, "twelveMonthsMotIncluded": false, "preReg": false, "heading": {"title": "Toyota Supra", "subtitle": "3.0 3dr", "__typename": "Heading"}, "attentionGrabber": "INVOICE FOR BUILD \u00a3115,000\ud83d\udca5", "rrp": null, "price": 89995, "priceGBX": 8999500, "priceExcludingFees": 89995, "priceExcludingFeesGBX": 8999500, "suppliedPrice": 89995, "suppliedPriceGBX": 8999500, "priceOnApplication": false, "plusVatIndicated": false, "vatStatus": null, "saving": null, "noAdminFees": true, "adminFee": null, "adminFeeInfoDescription": null, "dateOfRegistration": "2002-02-01", "homeDeliveryRegionCodes": null, "priceExcludingVat": 89995, "vatToBePaidUpfront": 0, "capabilities": {"marketExtensionHomeDelivery": null, "marketExtensionClickAndCollect": null, "marketExtensionCentrallyHeld": null, "marketExtensionOem": null, "digitalRetailing": {"enabled": true, "__typename": "DigitalRetailingCapability"}, "__typename": "Capabilities"}, "registration": "R****MJ", "generation": {"generationId": "c0b22e903d7a8a53114f48ba7c7a0aa0", "name": "Coupe (1993 - 1996)", "review": {"expertReviewSummary": null, "__typename": "GenerationReviewData"}, "__typename": "Generation"}, "hasShowroomProductCode": false, "isPartExAvailable": true, "isFinanceAvailable": true, "isFinanceFullApplicationAvailable": false, "financeProvider": "ZUTO", "financeDefaults": {"term": "48", "mileage": "10000", "depositAmount": "9000", "__typename": "FinanceDefaults"}, "hasFinanceInformation": false, "retailerId": "10044155", "privateAdvertiser": null, "advertiserSegment": "Independent", "dealer": {"dealerId": "10044155", "description": "", "distance": null, "stockLevels": {"atStockCounts": {"car": 8, "van": 2, "__typename": "AtStockCounts"}, "__typename": "DealerStockLevels"}, "assignedNumber": null, "atAwards": [], "branding": {"accreditations": [], "brands": [], "__typename": "DealerBranding"}, "capabilities": {"instantMessagingChat": null, "instantMessagingText": null, "__typename": "Capabilities"}, "reviews": {"numberOfReviews": "2", "overallReviewRating": "5.0", "__typename": "Reviews"}, "location": {"addressOne": "Unit 7, Rose Centre Retail Park, Nursery Drive", "addressTwo": "Norwich Road", "town": "North Walsham", "county": "Norfolk", "postcode": "NR28 0DR", "latLong": "52.8130746,1.3834347", "__typename": "Location"}, "marketing": {"profile": null, "strapline": "YOUR LOCAL SMALL ENGINE CAR SPECIALISTS", "__typename": "Marketing"}, "media": {"email": "yes", "dealerWebsite": null, "phoneNumber1": "(07822) 001329", "phoneNumber2": null, "protectedNumber": false, "__typename": "DealerMedia"}, "name": "North Walsham Car Sales", "servicesOffered": {"sellerPromise": null, "services": null, "products": ["APXV", "AT18FINANCECUST", "AT18SELLCUST", "ATCRTLCHK", "ATCVVAL", "ATMOVES", "BOOST", "CODRIVER", "DEAL", "DIGITALRETAIL", "DVID", "FPBARGAIN", "FPBONUS", "FPS", "ISP3VC", "ISSCUST", "ISS_PRODCAT", "PFIN", "PPI_3_STANDARD", "PROFILE", "RTLCHKPLUS", "STEX", "STOCKMANAGER", "VCHECKS", "VEHICLELOOKUP", "WATC", "WAV", "WDEW", "WDSV", "WIMT", "WKSP", "WLGE", "WMPS", "WSL"], "safeSelling": null, "videoWalkAround": null, "nccApproved": false, "isHomeDeliveryProductEnabled": false, "isPartExAvailable": true, "hasSafeSelling": false, "hasHomeDelivery": false, "hasVideoWalkAround": false, "__typename": "ServicesOffered"}, "__typename": "Dealer"}, "video": {"url": "https://youtube.com/embed/6ixADWeVg80?autoplay=1&rel=0", "preview": null, "__typename": "AdvertVideo"}, "spin": null, "imageList": {"nextCursor": null, "size": 79, "images": [{"url": "https://m.atcdn.co.uk/a/media/{resize}/bf03841908664eb7ac2f998042cc60c3.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Front Right", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/c1962490fa144c2a89ee44edf85ec4c8.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Side Right", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/7f93779e53b14b809d68ab1b5040be36.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Rear Right", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/14a4feb707d244ffb04979270be38577.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Rear", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/127dbcbee6564f349d7e5625bb2c19d7.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Rear", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/e396e8f62ae64515a43fcd35418bb902.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Rear Left", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/a69962b3022b4b1d8bf9085426f5f296.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Side Left", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/c058ca03df3346b790421eb44890077c.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Front Left", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/fcf1843c911c40dbb5c1be67a30198fa.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Front", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/fd04f207a3c24a2889885de99515a940.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Front Right", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/e755bc64e26c4fb4ae7da13cce21df99.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Wheel", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/63e3a01c31b44db3ada867919408d857.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Wheel", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/69dbc9b4d6b3462cbf34127ce10fe678.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Wheel", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/359e3d1a25494952b5b75c2adba545fc.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/a196b832832046868409506ad88db21f.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Wheel", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/4f2f14155f95445c80d0e524e90df428.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Front", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/04ad945546b4496a9ff8de685467e78d.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Rear Right", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/ede543c1af444021a54e3430acaa27f0.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Rear", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/00319595e9b64954a7eb597699b34cd7.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Inconclusive", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/abbea517202648609878d856c6668533.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Rear Left", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/8a83c58cefe54c979b0bbbbe2afc5981.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Seat Front Passenger", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/486f41d1892e4820895949370916c18b.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Door Interior", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/fd2b4640b6084847a51821313c23f894.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Seat Front Passenger", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/96f5982cad4f46c786c2e4013363a24f.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/8ce813bd2c564e848d9a8a5707c910da.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Boot", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/012a814e05c947719adfb13356641326.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Seat Driver", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/d2002acb7c814b70a33515d93ee9f567.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Door Interior", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/d1c6928362b34eafb667ada6923cd47e.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Seat Driver", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/3016231b4259495e8bd12ebe0ac17fa4.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Inconclusive", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/2a5f776bae0b458b9802b5a931bad751.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Seat Front Passenger", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/a4bd3f3ad9db4a84903e3ac3aeb6052f.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Misc Controls", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/38cb99d27a8542b3988be28012ca0c44.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Steering Wheel", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/1c68b39d3cc1462c903fcc7b32ea603a.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Driver Instruments", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/65e22389038a449398bd892bba837737.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/57b073d8423c4dc5977531f2c88a8a6f.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Misc Controls", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/ba3e1da065b54b6088af6938c6791bfa.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Driver Instruments", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/50410554070d4f69ac0502eee4b32158.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Misc Controls", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/a0c7725d4aa04f5489264699d0e8d842.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Interior Front", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/23c5fe56911d4cd6a1a586f3fb68539b.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Infotainment System", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/f2d1c636611f4fae93c069fd7c610293.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Infotainment System", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/26b64572f7134ebe8b5366bec9ad5696.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Misc Controls", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/2eaa31038210406e950d13cb9110c540.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Driver Instruments", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/3dc711db662645449276f5d8ef007747.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Misc Controls", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/c9f02fb4e0bf4ae8af3105bbbef837ac.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Misc Controls", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/280434bee48546eea797c93a9744bf9c.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Inconclusive", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/a993f757816c49c7a5b72357f97b3802.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Infotainment System", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/9e466603f86848218267508fbe94ee8d.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Misc Controls", "category": "Interior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/3e1200eefc0a443286edfab2a7dfde51.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Engine", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/fea434ee9534479a8bd77d5f696a66b8.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/40da270ee49840e68d439af58b0fd4e0.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/ec05c6aca53747689f47a84cb8346d6c.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Engine", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/16079009bfa449a8afbc3e7277d72f78.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Engine", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/1b79eab0b29f43aa8a40b6e188917b11.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/0aea5c91d4694227bc8c6ad36d6252af.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Inconclusive", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/7240a7e7d0444411a9bf1280dbe294ad.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Lights Rear", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/3a3d4769a7ac4423886ea5faa63d31de.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Rear Right", "category": "Exterior", "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/a1e976e68d7d463f873862b6d9b550b5.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/ea7bdc56036f4662ad3a6f5296b37a21.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/81367049f2f94facbd6b349af44bae0d.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/f04cb048eb4d4f4e962af97434c47163.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/0b7412cd2a5d4be9bcbe3f90c73565b2.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/62bbb9f688134d6f8bc02f2fdfc59f43.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/744b004a0c494f94b027cc81ed110072.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/75572e3527cc427ba3f948ec711f70b6.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/b974d67d9c6141d39555eae91d845473.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/782a8c76bc5a46ce98902258546451db.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/7e74a445bf2448b9a6d945029b644749.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/1f30254f14ca49e5b838250888a91041.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/ccd3b28796d1443bbb1abd0ca9bbb147.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/fc3e4b46d8124cd0881f15e832e181e8.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/e7686653455142b1a8c8d276702757d3.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/a9e7c73cf5ce4c64a2c6f3d2c323178e.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/5ee6258f57cd467c922b92c7d132af34.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/3865a239a9554be18b48b494963d0035.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/81d4d06cc75c4375a67cff96ae32819d.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/7cf3b1fb82154435a297292c71ebd9f1.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/ce1abac06b0a4ffb9a725f76adac511a.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/4e8605e45c4b41fc98e2ea6ab51ab856.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}, {"url": "https://m.atcdn.co.uk/a/media/{resize}/e4ef0fc1f9e645d0acff0586db60eb91.jpg", "templated": true, "autotraderAllocated": false, "classificationTags": [{"label": "Documentation", "category": null, "__typename": "ImageClassificationTag"}], "__typename": "ImageUrl"}], "__typename": "ImageList"}, "priceIndicatorRating": "NOANALYSIS", "priceIndicatorRatingLabel": "", "priceDeviation": null, "mileageDeviation": null, "mileage": {"mileage": 30000, "unit": "MILE", "__typename": "Mileage"}, "plate": "51", "year": 2002, "vehicleCheckId": "20567058e2f7454db1e418da314ed17c", "vehicleCheckStatus": "PASSED", "vehicleCheckSummary": {"type": "FULL", "title": "All checks passed", "performed": "2025-09-15", "writeOffCategory": null, "checks": [{"key": "STOLEN", "failed": false, "advisory": false, "critical": true, "warning": false, "__typename": "VehicleCheck"}, {"key": "SCRAPPED", "failed": false, "advisory": false, "critical": true, "warning": false, "__typename": "VehicleCheck"}, {"key": "WRITE_OFF_CATEGORY", "failed": false, "advisory": false, "critical": true, "warning": false, "__typename": "VehicleCheck"}, {"key": "IMPORTED", "failed": false, "advisory": false, "critical": true, "warning": false, "__typename": "VehicleCheck"}, {"key": "EXPORTED", "failed": false, "advisory": false, "critical": true, "warning": false, "__typename": "VehicleCheck"}, {"key": "HIGH_RISK", "failed": false, "advisory": false, "critical": true, "warning": false, "__typename": "VehicleCheck"}, {"key": "MILEAGE_DISCREPANCY", "failed": false, "advisory": false, "critical": true, "warning": false, "__typename": "VehicleCheck"}, {"key": "COLOUR_CHANGED", "failed": false, "advisory": false, "critical": true, "warning": false, "__typename": "VehicleCheck"}, {"key": "PRIVATE_FINANCE", "failed": false, "advisory": true, "critical": true, "warning": false, "__typename": "VehicleCheck"}, {"key": "TRADE_FINANCE", "failed": false, "advisory": false, "critical": false, "warning": false, "__typename": "VehicleCheck"}, {"key": "REGISTRATION_CHANGED", "failed": true, "advisory": false, "critical": true, "warning": true, "__typename": "VehicleCheck"}], "__typename": "VehicleCheckSummary"}, "sellerName": "North Walsham Car Sales", "sellerType": "Trade", "sellerProducts": ["APXV", "AT18FINANCECUST", "AT18SELLCUST", "ATCRTLCHK", "ATCVVAL", "ATMOVES", "BOOST", "CODRIVER", "DEAL", "DIGITALRETAIL", "DVID", "FPBARGAIN", "FPBONUS", "FPS", "ISP3VC", "ISSCUST", "ISS_PRODCAT", "PFIN", "PPI_3_STANDARD", "PROFILE", "RTLCHKPLUS", "STEX", "STOCKMANAGER", "VCHECKS", "VEHICLELOOKUP", "WATC", "WAV", "WDEW", "WDSV", "WIMT", "WKSP", "WLGE", "WMPS", "WSL"], "sellerLocation": "NORTH WALSHAM", "sellerLocationDistance": null, "sellerContact": {"phoneNumberOne": "(07822) 001329", "phoneNumberTwo": null, "protectedNumber": null, "byEmail": true, "__typename": "SellerContact"}, "description": "!!PLEASE SEE PICS FOR THE FULL ADVERT , ITS VERY LONG AND I CANT FIT IT ALL IN THE GIVEN SPACE !!\n\n1998 Factory Manual Twin turbo 6 speed vvti 2JZ-GTE \u2013 facelift. 900WHP.\n\nKey vehicle highlights:\n\u00d8\u00a0 1998 facelift VVTI Engine\n\u00d8\u00a0 Original colour Moonstone pearl Grey\u00a06NO\n\u00d8\u00a0 6 Speed Manual V161 Getrag gearbox OEM\n\u00d8\u00a0 SUPER low Mileage 30,000 miles from new.\n\u00d8\u00a0 Mileage since build 1250 miles\nSpec highlights:\n\u00d8\u00a0 Forged Race built engine 2JZ-GTE 3.2 Strokered.\n\u00d8\u00a0 Syvecs S7+ Plus ECU \u2013 fully mappable engine management system\n\u00d8\u00a0 RAYS Volk Racing Progressiv TE alloy wheels with brand new prielli p zero tyres all around. \n\u00d8\u00a0 HKS Coilover suspension\n\u00d8\u00a0 HKS Titanium racing exhaust(very loud!).\n\u00d8\u00a0 OEM front lip.\n\u00d8\u00a0 Carbon arm rest.\n\u00d8\u00a0 Carbon glove box.\n\u00d8\u00a0 The car has been in storage since 2008 and built over time it was recommissioned back to the road in 2021 with even more upgrades before my ownership.\nZero rust and in excellent condition. Was a grade 4.5 when imported in 2007.\n\u00d8\u00a0 MOT runs until November 2025 but will be sold with a fresh 12 months MOT .", "colour": "Grey", "manufacturerApproved": false, "insuranceWriteOffCategory": null, "owners": null, "keys": null, "vehicleCondition": null, "specification": {"driverPosition": "Unlisted", "isCrossover": false, "operatingType": null, "emissionClass": "Euro 1", "co2Emissions": null, "topSpeed": {"topSpeed": 155, "__typename": "Speed"}, "minimumKerbWeight": {"weight": 1490, "unit": "kg", "__typename": "Weight"}, "endLayout": null, "trailerAxleNumber": null, "bedroomLayout": null, "grossVehicleWeight": {"weight": null, "unit": null, "__typename": "Weight"}, "capacityWeight": {"weight": null, "unit": null, "__typename": "Weight"}, "liftingCapacity": {"weight": null, "unit": null, "__typename": "LiftingCapacity"}, "operatingWidth": {"width": null, "unit": null, "__typename": "OperatingWidth"}, "maxReach": {"length": null, "unit": null, "__typename": "MaxReach"}, "wheelbase": null, "berth": null, "bedrooms": null, "engine": {"power": {"enginePower": 326, "unit": "BHP", "__typename": "Power"}, "sizeLitres": 3, "sizeCC": 2997, "manufacturerEngineSize": 3, "__typename": "Engine"}, "exteriorWidth": {"width": null, "unit": null, "__typename": "ExteriorWidth"}, "exteriorLength": {"length": null, "unit": null, "__typename": "ExteriorLength"}, "exteriorHeight": {"height": null, "unit": null, "__typename": "ExteriorHeight"}, "capacityWidth": {"width": null, "unit": null, "__typename": "CapacityWidth"}, "capacityLength": {"length": null, "unit": null, "__typename": "CapacityLength"}, "capacityHeight": {"height": null, "unit": null, "__typename": "CapacityHeight"}, "seats": null, "axleConfig": null, "ulezCompliant": false, "doors": 3, "bodyType": "Coupe", "cabType": "Unlisted", "rawBodyType": "Coupe", "fuel": "Petrol", "transmission": "Manual", "style": null, "subStyle": null, "make": "Toyota", "model": "Supra", "trim": null, "optionalFeatures": null, "standardFeatures": null, "features": [], "battery": null, "techData": {"co2Emissions": null, "fuelConsumptionCombined": null, "fuelConsumptionExtraUrban": null, "fuelConsumptionUrban": null, "insuranceGroup": null, "minimumKerbWeight": "1490 kg", "zeroToSixtyMph": "4.9 seconds", "zeroToSixtyTwoMph": null, "cylinders": "6", "valves": "24", "enginePower": "326 bhp", "topSpeed": "155 mph", "engineTorque": "325.28 lbs/ft", "vehicleHeight": null, "vehicleLength": null, "vehicleWidth": null, "wheelbase": null, "fuelTankCapacity": "70 litres", "grossVehicleWeight": null, "luggageCapacitySeatsDown": null, "bootspaceSeatsUp": null, "vehicleWidthInclMirrors": null, "maxLoadingWeight": null, "standardFeatures": [], "chargingData": null, "__typename": "TechData"}, "annualTax": {"standardRate": null, "__typename": "AnnualTax"}, "oemDrivetrain": null, "bikeLicenceType": null, "derivativeId": "1c89c64eef2343d884f409c3367eff8b", "frameSizeCM": null, "frameMaterial": null, "frameStyle": null, "suspensionType": null, "gearShifter": null, "brakeType": null, "motorMake": null, "chargeTimeMinutes": null, "numberOfGears": null, "tyreDiameterInches": null, "driveTrain": "Rear Wheel Drive", "torque": {"torque": 441, "unit": "Nm", "__typename": "Torque"}, "range": null, "interiorDescription": null, "exteriorDescription": "Grey exterior colour", "__typename": "Specification"}, "stockType": "physical-stock", "condition": "Used", "finance": null, "locationArea": null, "reservation": {"status": null, "eligibility": "ELIGIBLE", "feeCurrency": "GBP", "feeInFractionalUnits": 9900, "__typename": "AdvertReservation"}, "__typename": "Advert"}

2025-12-05 12:35:46 [asyncio] ERROR: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fba954e4890>

2025-12-05 12:35:46 [asyncio] ERROR: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fba954e50d0>

2.11.2

127.0.0.1:6030

cb592ec17ab3a8d0

2025-12-05 12:35:44 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://www.autotrader.co.uk/at-graphql> (referer: https://www.autotrader.co.uk/at-gateway?opname=SearchResultsListingsGridQuery&opname=SearchResultsFacetsWithGroupsQuery) ['zyte-api']

2025-12-05 12:35:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.autotrader.co.uk/at-graphql>

2025-12-05 12:34:47 [scrapy.extensions.logstats] INFO: Crawled 8 pages (at 0 pages/min), scraped 7 items (at 0 items/min)

2025-12-05 12:28:47 [scrapy.utils.log] INFO: Scrapy 2.11.2 started (bot: SourcingV2)
2025-12-05 12:28:47 [scrapy.utils.log] INFO: Versions: lxml 5.2.2.0, libxml2 2.12.6, cssselect 1.2.0, parsel 1.9.1, w3lib 2.1.2, Twisted 24.3.0, Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.7, Platform Linux-5.15.0-1098-azure-x86_64-with-glibc2.36
2025-12-05 12:28:47 [auto_selling_autotrader.co.uk] INFO: Starting auto_selling_autotrader.co.uk spider
2025-12-05 12:28:47 [auto_selling_autotrader.co.uk] INFO: _job: 307506d8d1c111f0b120928152083da5, SCRAPY_JOB: None
2025-12-05 12:28:47 [scrapy.addons] INFO: Enabled addons:
[]
2025-12-05 12:28:47 [asyncio] DEBUG: Using selector: EpollSelector
2025-12-05 12:28:47 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor
2025-12-05 12:28:47 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.unix_events._UnixSelectorEventLoop
2025-12-05 12:28:47 [scrapy.extensions.telnet] INFO: Telnet Password: cb592ec17ab3a8d0
2025-12-05 12:28:47 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.feedexport.FeedExporter',
 'scrapy.extensions.logstats.LogStats',
 'scrapy.extensions.closespider.CloseSpider']
2025-12-05 12:28:47 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'SourcingV2',
 'CLOSESPIDER_TIMEOUT': 3600,
 'CONCURRENT_REQUESTS': 5,
 'FEED_EXPORT_ENCODING': 'utf-8',
 'LOG_FILE': '/var/log/scrapyd/logs/sourcing_v2/auto_selling_autotrader.co.uk/307506d8d1c111f0b120928152083da5.log',
 'LOG_FORMATTER': 'crawlers.log_formatter.SourcingLogFormatter',
 'NEWSPIDER_MODULE': 'spiders',
 'REQUEST_FINGERPRINTER_CLASS': 'scrapy_zyte_api.ScrapyZyteAPIRequestFingerprinter',
 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
 'SPIDER_MODULES': ['spiders', 'auth_check'],
 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor',
 'USER_AGENT': ''}
2025-12-05 12:28:47 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2025-12-05 12:28:47 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2025-12-05 12:28:47 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy_zyte_api.ScrapyZyteAPIDownloaderMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2025-12-05 12:28:47 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Setting up IdGenerationMiddleware for auto_selling
2025-12-05 12:28:47 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy_zyte_api.ScrapyZyteAPISpiderMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware',
 'crawlers.middlewares.photo_download_middleware.PhotoDownloadMiddleware',
 'crawlers.auto_selling.middlewares.id_gen_middleware.IdGenMiddleware']
2025-12-05 12:28:47 [scrapy.middleware] INFO: Enabled item pipelines:
['crawlers.pipelines.post_to_azurebus_auto_selling.AzureBusPipelineAutoSelling',
 'crawlers.pipelines.save_to_cdlq_db_auto_selling.SaveToCdlqDbAutoSelling']
2025-12-05 12:28:47 [scrapy.core.engine] INFO: Spider opened
2025-12-05 12:28:47 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_CLIENT_ID, AZURE_TENANT_ID
2025-12-05 12:28:47 [common.service_bus.service_bus_client_factory] INFO: ServiceBusClient initialized via DefaultAzureCredential for alx-production-bus.servicebus.windows.net
2025-12-05 12:28:47 [crawlers.pipelines.base_azure_bus_pipeline] INFO: [AzureBusPipeline] Connected to topic auto.sourcing-v2
2025-12-05 12:28:47 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2025-12-05 12:28:47 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6030
2025-12-05 12:28:47 [scrapy-playwright] INFO: Starting download handler

2025-12-05 12:28:47 [scrapy-playwright] INFO: Starting download handler
2025-12-05 12:28:52 [auto_selling_autotrader.co.uk] INFO: Scraper started with retailer ID: 10044155 and multipost_profile_id: 33ef9f8c-2288-413a-ac07-b46f824492e5 and job_id: 307506d8d1c111f0b120928152083da5 and url: https://www.autotrader.co.uk/cars/retailer/stock?advertising-location=at_cars&advertising-location=at_profile_cars&onesearchad=Used&onesearchad=Nearly%20New&onesearchad=New&postcode=E17%208XP&retailerId=10044155&sort=price-asc
2025-12-05 12:28:52 [auto_selling_autotrader.co.uk] INFO: Getting list for retailer ID: 10044155, page: 1
2025-12-05 12:28:52 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2025-12-05 12:28:53 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://www.autotrader.co.uk/at-gateway?opname=SearchResultsListingsGridQuery&opname=SearchResultsFacetsWithGroupsQuery> (referer: None) ['zyte-api']
2025-12-05 12:28:53 [auto_selling_autotrader.co.uk] DEBUG: response: <200 https://www.autotrader.co.uk/at-gateway?opname=SearchResultsListingsGridQuery&opname=SearchResultsFacetsWithGroupsQuery>
2025-12-05 12:28:53 [auto_selling_autotrader.co.uk] INFO: Page: 1/1, total results: 8
2025-12-05 12:28:53 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_CLIENT_ID, AZURE_TENANT_ID
2025-12-05 12:28:53 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 12:28:53 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 12:28:53 [azure.identity._credentials.chained] INFO: DefaultAzureCredential acquired a token from WorkloadIdentityCredential
2025-12-05 12:28:53 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxsourcingstorageprod.table.core.windows.net:443
2025-12-05 12:28:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 12:28:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202512048345502%27 HTTP/1.1" 200 None
2025-12-05 12:28:53 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202512048345502: 1
2025-12-05 12:28:53 [auto_selling_autotrader.co.uk] INFO: Getting details for advert ID: 202512048345502, item: 202512048345502
2025-12-05 12:28:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 12:28:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202511288197013%27 HTTP/1.1" 200 None
2025-12-05 12:28:53 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202511288197013: 1
2025-12-05 12:28:53 [auto_selling_autotrader.co.uk] INFO: Getting details for advert ID: 202511288197013, item: 202511288197013
2025-12-05 12:28:53 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2025-12-05 12:28:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 12:28:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202511268150277%27 HTTP/1.1" 200 None
2025-12-05 12:28:54 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202511268150277: 1
2025-12-05 12:28:54 [auto_selling_autotrader.co.uk] INFO: Getting details for advert ID: 202511268150277, item: 202511268150277
2025-12-05 12:28:54 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2025-12-05 12:28:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 12:28:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202511258121765%27 HTTP/1.1" 200 None
2025-12-05 12:28:54 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202511258121765: 1
2025-12-05 12:28:54 [auto_selling_autotrader.co.uk] INFO: Getting details for advert ID: 202511258121765, item: 202511258121765
2025-12-05 12:28:54 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2025-12-05 12:28:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 12:28:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202511187961945%27 HTTP/1.1" 200 None
2025-12-05 12:28:54 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202511187961945: 1
2025-12-05 12:28:54 [auto_selling_autotrader.co.uk] INFO: Getting details for advert ID: 202511187961945, item: 202511187961945

             'OriginUrl': 'https://m.atcdn.co.uk/a/media/8767c03b68434135954c34be8f974c60.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_2.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 502493,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/33458e21041748bd8075557111accd51.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_3.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 489495,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/2d84266d3cfb4ee5abb221d60769da33.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_4.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 395298,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/96d2aa49e3b04765a5e43837f816bc60.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_5.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 416660,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/6d9e23484920434c8a16eec1674973dc.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_6.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 503484,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/70a00b09efc44b719bf2b1b7d7ec00b5.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_7.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 505611,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/21ffad6c884a4ea983436cd7ec6d6536.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_8.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 515039,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/3168b76ffe26429cb1191343e8c58a28.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_9.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 473397,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/dcc6417f601a4e02afaabcf51d340bb3.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_10.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 427908,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/339981f7fd1f42a883d063cbc0aea132.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_11.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 424657,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/fbbfca6d88b24206b02f6bf61dfe46c9.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_12.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 463620,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/465de8fa501a45acb4adc6969c025172.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_13.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 467385,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/f0e92c5ba9db426b9d660e12772a46e3.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_14.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 221549,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/e95544e6a9624c2ba839326bb1da5aef.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_15.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 278122,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/fef3e8be8b3a4d03a19f5196619abdf1.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_16.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 265418,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/7c8565b3ff9e4a3590308d4618057dfa.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_17.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 244203,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/0da4e4ea423b47268a0756232f83631b.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_18.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 180213,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/9a47441f8b364d63a0fe20697d24d2a0.jpg',
             'Width': 1600},
            {'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bc27be68-26a3-50dd-b638-69be193efc53/bc27be68-26a3-50dd-b638-69be193efc53_image_19.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 233835,
             'Height': 1200,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/d39dbecdc5ac4467b2a20ab8680df141.jpg',
             'Width': 1600}],
 'price': 15695,
 'price_includes_vat': True,
 'registration_date': '2017-07-17',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Audi Q5 2.0 TDI Sport S Tronic quattro Euro 6 (s/s) 5dr',
 'transmission': 'automatic',
 'trim': 'Sport',
 'year': 2017}
2025-12-05 12:35:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 12:35:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202510046885777%27 HTTP/1.1" 200 None
2025-12-05 12:35:45 [auto_selling_autotrader.co.uk] INFO: Saving data for 202510046885777: {'created_time': 1764938145.608785, 'last_price_update_time': 1764938145.608795}
2025-12-05 12:35:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 12:35:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202510046885777') HTTP/1.1" 204 0
2025-12-05 12:35:45 [scrapy.core.engine] INFO: Closing spider (finished)
2025-12-05 12:35:45 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close cursor: 'SaveToCdlqDbAutoSelling' object has no attribute 'cursor'
2025-12-05 12:35:45 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close connection: 'SaveToCdlqDbAutoSelling' object has no attribute 'conn'
2025-12-05 12:35:45 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2025-12-05 12:35:45 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2025-12-05 12:35:45 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process scrape complete CloudEvent for item 307506d8d1c111f0b120928152083da5
2025-12-05 12:35:45 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2025-12-05 12:35:45 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2025-12-05 12:35:45 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2025-12-05 12:35:45 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2025-12-05 12:35:45 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2025-12-05 12:35:45 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.MAPPED: 3> -> <SessionState.END_SENT: 4>
2025-12-05 12:35:45 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.OPENED: 9> -> <ConnectionState.CLOSE_SENT: 11>
2025-12-05 12:35:45 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.CLOSE_SENT: 11> -> <ConnectionState.END: 13>
2025-12-05 12:35:45 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.END_SENT: 4> -> <SessionState.DISCARDING: 6>
2025-12-05 12:35:45 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2025-12-05 12:35:45 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2025-12-05 12:35:45 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2025-12-05 12:35:45 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2025-12-05 12:35:45 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2025-12-05 12:35:45 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: [AzureBusPipeline] connection closed
2025-12-05 12:35:45 [auto_selling_autotrader.co.uk] INFO: auto_selling_autotrader.co.uk Crawl ended with reason finished, scrape types: {<ScrapeType.NEW: 1>: 8, <ScrapeType.NEW_DUPLICATE_ID: 4>: 0, <ScrapeType.PRICE_UPDATE: 2>: 0, <ScrapeType.AUCTION_UPDATE: 3>: 0, <ScrapeType.SKIPPED: 0>: 0}
2025-12-05 12:35:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 12:35:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20last_price_update_time%20lt%201763642146 HTTP/1.1" 200 None
2025-12-05 12:35:46 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (8 items) in: file:///var/lib/scrapyd/items/sourcing_v2/auto_selling_autotrader.co.uk/307506d8d1c111f0b120928152083da5.jl
2025-12-05 12:35:46 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 70000,
 'downloader/request_count': 9,
 'downloader/request_method_count/POST': 9,
 'downloader/response_bytes': 431724,
 'downloader/response_count': 9,
 'downloader/response_status_count/200': 9,
 'elapsed_time_seconds': 418.834127,
 'feedexport/success_count/FileFeedStorage': 1,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2025, 12, 5, 12, 35, 46, 247563, tzinfo=datetime.timezone.utc),
 'item_scraped_count': 8,
 'log_count/DEBUG': 463,
 'log_count/ERROR': 2,
 'log_count/INFO': 281,
 'memusage/max': 145346560,
 'memusage/startup': 126439424,
 'request_depth_max': 1,
 'response_received_count': 9,
 'scheduler/dequeued': 9,
 'scheduler/dequeued/memory': 9,
 'scheduler/enqueued': 9,
 'scheduler/enqueued/memory': 9,
 'scrapy-zyte-api/429': 0,
 'scrapy-zyte-api/attempts': 16,
 'scrapy-zyte-api/error_ratio': 0.4375,
 'scrapy-zyte-api/errors': 7,
 "scrapy-zyte-api/exception_types/<class 'aiohttp.client_exceptions.ClientConnectorError'>": 7,
 'scrapy-zyte-api/fatal_errors': 0,
 'scrapy-zyte-api/mean_connection_seconds': 13.417265391106614,
 'scrapy-zyte-api/mean_response_seconds': 13.459644764804075,
 'scrapy-zyte-api/processed': 9,
 'scrapy-zyte-api/request_args/customHttpRequestHeaders': 9,
 'scrapy-zyte-api/request_args/experimental.requestCookies': 8,
 'scrapy-zyte-api/request_args/experimental.responseCookies': 9,
 'scrapy-zyte-api/request_args/httpRequestBody': 9,
 'scrapy-zyte-api/request_args/httpRequestMethod': 9,
 'scrapy-zyte-api/request_args/httpResponseBody': 9,
 'scrapy-zyte-api/request_args/httpResponseHeaders': 9,
 'scrapy-zyte-api/request_args/url': 9,
 'scrapy-zyte-api/status_codes/0': 7,
 'scrapy-zyte-api/status_codes/200': 9,
 'scrapy-zyte-api/success': 9,
 'scrapy-zyte-api/success_ratio': 1.0,
 'scrapy-zyte-api/throttle_ratio': 0.0,
 'start_time': datetime.datetime(2025, 12, 5, 12, 28, 47, 413436, tzinfo=datetime.timezone.utc)}
2025-12-05 12:35:46 [scrapy.core.engine] INFO: Spider closed (finished)
2025-12-05 12:35:46 [asyncio] ERROR: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fba954e4890>
2025-12-05 12:35:46 [asyncio] ERROR: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fba954e50d0>

/1/log/utf8/sourcing_v2/auto_selling_autotrader.co.uk/307506d8d1c111f0b120928152083da5/?job_finished=True

http://scrapyd-1:6800/logs/sourcing_v2/auto_selling_autotrader.co.uk/307506d8d1c111f0b120928152083da5.log