| Rev |
Age |
Author |
Path |
Log message |
Diff |
| 37448 |
13 d 12 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Daily re-check for all brands; Oppo back to parallel pools
Two changes.
1. Re-check window 4 days (secondary) and 2 days (tertiary) -> 1 day everywhere.
Daily demand becomes the full universe rather than a fraction of it:
Oppo 4,798 + 4,040 = 8,838/day
Vivo 9,407 + 675 = 10,082/day (doing 15,345 -- fine)
Realme 1,973 + 1,081 = 3,054/day
2. Oppo's two pools run in PARALLEL again, reverting the merge in r37447 for that
brand only. Realme stays merged.
The merge was a straight trade of throughput for memory and oppo could not
afford it. Measured over 32 minutes and again over an hour the next morning:
3,555 then 3,456/day against 5,280 before merging. Batch cadence settled at a
very regular ~15.5 min per cycle, so a 30-imei merged batch takes ~10.5 min =
~21s/imei, against the 10.2s it managed unmerged. At 21s the ceiling is
86400/21 = 4,114/day even with zero idle, so no batch size and no shorter
fixedDelay could have reached 8,838. Serialising simply costs more per imei
here than running two browsers does.
Realme keeps the merge: it needs 3,054/day and delivers 2,952 merged, so a
small size bump covers it without a second browser.
Sizes: oppo 25 per pool (2 jobs in parallel), realme 12+12 merged, vivo 50+10
unchanged. Vivo has already cleared its entire secondary backlog -- the pool
reads 0 and both lists come back empty -- which is what the batch of 50 was for.
Cost: oppo goes back to two concurrent drivers, so the fleet is 3 rather than 2,
roughly +700MB. Acceptable against the ~2GB freed today by reaping orphaned
browsers and capping retries, but it is the reason realme was left merged.
Sizes are a starting point, not a final answer: oppo's per-imei time differs
markedly between merged and parallel modes, so re-measure before tuning further. |
|
| 37447 |
14 d 3 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Selenium: one browser per brand instead of one per pool
Oppo and Realme each ran secondary and tertiary as separate @Scheduled jobs, so
each brand opened two ChromeDrivers at once and the fleet ran four. Each driver
tree costs roughly 850MB. This box co-hosts a 9.4GB tomcat with ~3GB available
and has been OOM-killed twice this month -- tomcat was the victim both times, so
peak concurrency is the thing that matters here.
Combined into one job per brand. Nothing downstream changes: the two pools differ
only in which named query fills them, and both already funnel into the same
updateActivationDate -> checkWarranty -> saveActivation path. They are disjoint by
construction (secondary excludes anything with a FofoLineItem, tertiary is
FofoLineItem-based); distinct() is insurance, not a fix for a known overlap.
Sizing matters, because merging SERIALISES work that used to run in parallel and
keeping the old batch sizes would quietly cost throughput. Measured post-cap at
10.2s/imei (oppo, down from 14.6 after r37445) and 14.2s/imei (realme), solving
M * 86400 / (300 + M*t):
oppo 2 parallel jobs x10 = 4,299/day -> merged 15+15 = 4,277/day (parity)
realme 2 parallel jobs x10 = 3,910/day -> merged 10+10 = 2,959/day (-24%)
Oppo is sized to hold parity because it is already short of its 4,798/day need.
Realme is left at 20 -- it needs 2,243/day, so it can absorb the dip in exchange
for shorter batches and a shorter-lived browser.
What this saves and does not save: total driver-SECONDS are roughly unchanged,
which is the point of resizing. PEAK concurrent drivers halves from 4 to 2.
Also skips starting a browser at all when both pools come back empty -- currently
never true, but it costs nothing and a browser launched to do nothing is pure
waste on this box.
checkOppoImeiStatus/Tertiary and the realme equivalents are left in place for
manual invocation; they are simply no longer scheduled. |
|
| 37395 |
16 d 4 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
observability: retire the dead Nagios monitoring path, migrate balance signals to Micrometer
There is no Nagios server and the NRPE daemons have been removed from every
host, so the properties files written under /var/log/services were being
produced for nobody.
Deleted:
- CronMetricsService: zero references. Spring instantiated the @Service so its
constructor registered three meters, but nothing ever incremented them --
cron_execution_* read 0 in prometheus while cron_job_count_total (from
CronJobMonitorAspect) had recorded 1,597 executions.
- NagiosMonitoringScheduledSkeleton.
Removed the nagios properties writes from ScheduledSkeleton (-155 lines) and
KnowlarityCallMonitorScheduler (-46, including its now-purposeless
markTaskComplete helper). These recorded a per-job 0/1 status that
CronJobMonitorAspect already captures properly as success, failure and
duration metrics.
Kept the two signals that are genuinely useful: SMS gateway balance and the
ThinkWalnut recharge wallet balance now publish as Micrometer gauges via the
new BalanceGauges component, scraped from /actuator/prometheus. -1 means 'not
read yet' so a scrape before the first run is distinguishable from a real zero.
No behaviour change to any scheduled job. Compiles clean. |
|
| 37292 |
30 d 8 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Hot deals: javadoc matches reindex-based sync
syncHotDealFlags no longer mirrors model_hot_deal into tag_listing.hot_deals;
since r37276 hot_deal_b is derived from the active window at index time, so the
daily job re-indexes models whose window opened today or closed yesterday.
Comment-only, no behaviour change. |
|
| 37258 |
35 d 7 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Schedule syncHotDealFlags daily 00:20 (ScheduledSkeleton) + --syncHotDealFlags CLI arg: completes r37255 - without the skeleton trigger the tag_listing.hot_deals mirror never runs, so deal windows would not start/expire in Solr |
|
| 37234 |
37 d 5 h |
vikas |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Updated Gmail Timeout |
|
| 37230 |
37 d 5 h |
vikas |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Updated Gmail Timeout |
|
| 37172 |
42 d 11 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
PJP agenda instances: nightly 23:45 sync cron + manual run arg |
|
| 37162 |
42 d 13 h |
vikas |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Shopify Inventory Sync |
|
| 37159 |
43 d 7 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Nightly partner identity integrity monitor (7 violation classes over fofo_store/user.user/user_accounts/user_role incl. same-GST/mobile multi-open-store mandate checks); scheduled 07:00 + CLI trigger partnerIdentityIntegrityMonitor |
|
| 37083 |
58 d 10 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Wire noon price-drop cron to per-drop batch (PriceDropBatchService)
@Scheduled("0 0 12") now delegates to
BatchScheduledTasks.reprocessPriceDropsWithBatch() -> PriceDropBatchService,
so each price drop commits in its own REQUIRES_NEW transaction instead of one
~20-min transaction holding user_wallet locks and timing out live
wallet/order/scan traffic at noon.
Requires profitmandi-dao PriceDropBatchService / PriceDropProcessingHelper. |
|
| 37040 |
63 d 11 h |
ranu |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
on billing delay partner map to focus and revival |
|
| 36993 |
70 d 11 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Thread dryRun through sendCreditNote; --sendCreditNote defaults to dry-run (--live for real); add 06:00-on-1st monthlyMarginsCnDryRun scheduled job |
|
| 36552 |
118 d 5 h |
ranu |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
changeList |
|
| 36511 |
123 d 8 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Add daily inventory snapshot reconciliation cron - syncs currentinventorysnapshot with scan truth for active tag_listing items - emails drift report via googleMailSender to sdtech - runs daily at 5:30 AM, CLI trigger: --reconcileInventorySnapshot |
|
| 36361 |
141 d 4 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Switch @Scheduled updatePartnerLimit to batch-tracked version. Was calling legacy scheduledTasks.updatePartnerLimit (one synchronized tx, writes to all ~1500 partners each run, no audit). Now calls batchScheduledTasks.updatePartnerLimitWithBatch (writes only changed partners in per-partner REQUIRES_NEW, records each run in cron_batch / cron_batch_item, sends failure email on partial failures). Same cadence (every 20 min), same business logic. Runs are now visible in /admin/cron-batches UI. |
|
| 36292 |
146 d 11 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Reduce DB scan pressure from runaway IMEI and IRN crons
- IRN cron (updateIrnsToInvoices): fixedDelay 2s -> 5s; migration cron
has finished but keeps full-scanning 821k orders every 2s for 0 rows
(every billed order already has irnGenerated populated true/false).
- IMEI activation crons: fixedDelay 60s -> 5min across all 6 methods
(vivoImeiActivation, vivoImeiActivationTertiary, oppo, oppoTertiary,
realme, realmeTertiary). Each was scanning orders since 2021-01-01
every minute; 5min is well within activation-latency tolerance. |
|
| 36253 |
151 d 14 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Separate secondary/tertiary IMEI activation crons for Vivo/Oppo/Realme, perf fixes: shared saveActivation, Response leak fixes, /tmp cleanup, OpenCV static init, early break, remove class-level @Transactional from StandAlone |
|
| 36113 |
163 d 7 h |
ranu |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
rbm rating consolidate mail |
|
| 36065 |
169 d 4 h |
aman |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Fix:Mail error |
|