| Rev |
Age |
Author |
Path |
Log message |
Diff |
| 37614 |
28 d 3 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/migrations/ |
Run each internal GRN in a transaction that is actually applied
The internal GRN one-off failed immediately with "no transaction is in progress", thrown while
flushing the Hibernate session at commit.
The per invoice method was annotated to start its own transaction, but it sat in the same bean as
the loop that called it. Spring applies @Transactional through a proxy, and a call from one method
of a bean to another never leaves the object, so the annotation was inert - the receiving ran with
no transaction at all while the driver had suspended the surrounding one. The repositories still
bound a session to the thread, and the flush at commit then found nothing to flush into.
The receiving moves to its own bean, so the driver now reaches it through the proxy and the
transaction is real. The driver keeps no transaction of its own, which is what lets one invoice
fail without disturbing those already received.
No change to what is received or to the conditions under which an invoice is skipped. |
|
| 37613 |
28 d 4 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Receive internal transfer stock that was billed but never GRNed
Stock moved between our own warehouses is dispatched against an internal purchase order and
billed, but the destination warehouse still has to receive it by hand through the portal. Where
that never happened the units exist on an invoice and nowhere in inventory, and there is no way
to clear a backlog of them short of keying each invoice in again.
This adds a cron one-off that receives them. Per invoice it builds the rows the Excel GRN upload
would have carried and hands them to PurchaseOrderService.addPORowModels, which creates the
supplier invoice, the purchase and the inventory items in a single call, so none of receiving is
reimplemented here - the portal and this take the same path and can only ever agree.
It is deliberately narrow about what it will touch. Only INTERNAL buyers, because nothing should
be able to receive a partner's goods on their behalf. Only invoices with no supplier invoice
already recorded, so a repeat run skips what it has already done rather than receiving twice.
Only invoices resolving to a single purchase order, since addPORowModels resolves one order for
the whole map it is given and would otherwise attribute an invoice to the wrong one - for the
same reason each invoice is passed on its own. A serialized line whose serial number is missing
aborts its invoice instead of creating a unit no scan could ever match.
Each invoice commits or rolls back on its own, so one bad invoice cannot undo the ones already
received, and a dry run reports what it would receive without writing anything.
The receipt is recorded against the destination warehouse, which is what warehouse inventory is
keyed by. The buyer on the order is read only to confirm the store is internal; partner side
inventory is a separate receipt and is not touched. |
|
| 37602 |
29 d 17 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Remove no-op refreshSnapshotAgeing task; restore originalInventoryItemId backfill call
refreshSnapshotAgeing ran every 30 minutes aggregating warehouse.inventoryItem.rootInvoiceDate,
a column nothing ever wrote, so its UPDATE ... JOIN matched zero rows on every run and
currentinventorysnapshot.oldest_invoice_date was never populated on any of 11,686 rows.
Removed along with the sessionFactory field it was the only consumer of.
Application.java had migrations.migrateWarehouseOriginalInventoryItemId(batchSize) commented
out while still logging 'Starting migration...' and 'Migration completed.', so
--migrateOriginalInventoryItemId reported success while doing nothing. Restored the call;
it remains opt-in via the CLI flag and cannot fire on its own.
Requires profitmandi-dao r37601 (entity fields removed). |
|
| 37595 |
30 d 12 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Move the full partner-limit pass from every 20 minutes to daily
The 2-minute sweep already recalculates the limit for every partner whose base
moved, which covers investment and utilisation - the inputs that actually change.
The full 980-partner rescan only exists for inputs base cannot see: credit risk,
the SIDBI floor and hard_limit.
Those move on a far slower clock. RISK_DECREASE_DAYS_THRESHOLD is 90 days, and
transaction.fofo_sidbi_sanction has had no new row since Oct 2024 and no recorded
settlement. Running the rescan 72 times a day to catch them was 71 wasted passes.
Folded into the existing daily job, after the aged-stock refresh so the limit pass
sees haircuts for partners whose Apple or demo stock crossed its threshold
overnight. Leaves two scheduled jobs instead of three.
Limit latency is unaffected - that is the 2-minute sweep's job and it is unchanged
(~3 partners per window show a changed base, peak 18). The --updatePartnerLimitWithBatch
CLI entrypoint is untouched for manual runs. |
|
| 37589 |
30 d 14 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Drive credit limits off the investment sweep; stop phantom limit writes
Adds the 2-minute sweep schedule and points the limit recalculation at the
partners whose base_value actually moved, instead of rescanning all 980 every
20 minutes. Measured on production: 3 partners change per 2-minute window on
average, peak 18.
- ScheduledSkeleton: 2-min sweepPartnerInvestment, daily refreshAgedStockDaily
at 00:20; the 20-min full pass is kept as a backstop for inputs base does not
capture (risk, the SIDBI floor, hard_limit)
- BatchScheduledTasks.sweepPartnerInvestment: sweep, then limits for changed
bases only
- calculateChangedPartnerLimits(restrictTo) to support that
- scaleMoney(): round to paise before comparing and storing. The limit comes out
of a double multiply, so 65123.7400 round-tripped as 65123.740000000005 and
compareTo called it a change. 208 of 230 'changed' partners in one production
run differed by under half a paisa, rewriting sd_credit_requirement and
dtr.credit_account ~14,000 times a day for identical values and churning the
SIDBI mirror. Real changes were never below a rupee, so 2dp cannot suppress one.
- getFirstBillingDates: one grouped query instead of one per partner
Credit limits for 24 partners will rise on the first sweep after deploy - that is
the aged-Apple double-deduction correction landing (see r37588), not a defect.
NOTE: cron is a standalone Spring Boot jar; the new schedules only fire once it
is restarted with --spring.profiles.active=scheduled. |
|
| 37569 |
31 d 19 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
fcm: stop holding a row lock on the whole batch across every FCM call
sendNotification lived in ScheduledTasks, which carries a class-level @Transactional,
so one pass was one transaction: it loaded EVERY pending row uncapped, then per row
made an FCM call over the network and stamped sentTimestamp on the entity. A dirty
row stays write-locked until commit, so a run held a lock on every notification in
the batch for the SUM of all the remote waits. Volume is real -- 11,581 sent on
2026-09-09, 12,036 on 2026-09-05 -- and HttpClientFactory sets a 10s socket timeout,
so one bad run could hold thousands of locks for minutes.
That is the shape behind a 33 SECOND average InnoDB row-lock wait server-wide
(Innodb_row_lock_time 63,291s over 15.4 days uptime, max 51,941ms against a 50s
innodb_lock_wait_timeout -- which is where LockAcquisitionException comes from).
Moved to PushNotificationSendService using the same seam the invoicing pass already
uses in InvoiceService.updateIrnsToInvoices: one short read transaction to pick up
the batch and build the payloads, NO transaction across the network call, and one
REQUIRES_NEW transaction per row to record the outcome. No lock is held while waiting
on FCM.
It is a separate bean, and reaches its own transactional methods through an injected
self-reference, because REQUIRES_NEW is applied by a Spring proxy and a plain
self-invocation would bypass it -- same idiom, for the same reason, as InvoiceService.
Also caps the batch at 500/run (30,000/hour against a 12,000 peak day) using the
already-present but never-wired selectPendingNotifications(limit); the previous query
was uncapped, which is what let a campaign burst become one multi-minute transaction.
Send outcomes are unchanged: 200 stamps now, anything else stamps the 1970 sentinel.
The HttpClient is now closed, which it previously was not. |
|
| 37567 |
31 d 20 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Schedule the catalogue delist daily at 05:30
Runs 30 minutes before the existing 06:00 scheduledPushDataToSolr in the same
class, so the same morning's reindex publishes it. A bulk UPDATE fires no
TagListingChangeListener event, so without that ordering the portal would lag
until 18:00.
Gated on catalog.autoDelist.enabled with an inline default of true - env profiles
have no fallback between each other, so a key missing from one profile's
properties would otherwise stop the context booting.
Also exposes --delistDeadListings for a manual run, honouring the same flag. |
|
| 37565 |
32 d 5 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
IMEI activation: 20s tick lanes for oppo/realme/vivo, idle once the day clears
Replaces the two once-a-day passes with chunk-per-tick lanes. Each tick takes one
chunk and returns; when a brand's pool comes back empty its turn is skipped, and
when everything is clear the ticks do nothing and stay silent until midnight.
- browser lane (StandAlone): one chunk of 25 for one brand every 20s, oppo and
realme by turns. Still exactly one ChromeDriver alive at a time.
- vivo lane: its own tick, chunk of 250. It must not share the browser lane --
0.24s an imei against oppo's 10.2s means it would need ~70 hours behind them
for work it does alone in 39 minutes.
The snapshot is gone; the pool query is the cursor. That is what makes a restart
cost one chunk instead of the day: the 12:03 restart on 09-Sep forfeited ~4,000
lookups and the whole afternoon, and last_finish had read -1 for three days.
The snapshot existed to stop the re-ask loop (realme, 29-Aug: 4,524 requests
against 1,004 distinct imeis). That is now closed at the source instead -- oppo,
realme and motorola stamp every imei they asked about, not just the ones that
produced a map entry, so a failed lookup rests until tomorrow rather than coming
back on the next tick. Motorola is fixed pre-emptively; nothing schedules it yet.
Also:
- secondary and tertiary are merged by turns rather than concatenated. Safe while
a pass walked to the end; without that guarantee oppo's 163 tertiary serials sat
behind 3,819 secondary ones and would only be reached on a day that cleared.
- vivo abandons a tick rather than the chunk when the captcha solver returns no
code -- one probe per 20s while it is down instead of 250, and no rows rested
over a transient outage.
- the funnel gauges move from a pass to a day. due is measured on the first tick
after midnight, the rest accumulate, and last_finish_epoch becomes a real
completion clock. Truncation is detected at the midnight rollover, which is the
case that never reaches an end-of-run at all.
This does not create capacity. At the 21s/imei measured on 09-Sep the pool still
needs ~35 hours and will not clear; it now rolls over visibly instead of silently.
The lever for that is DAYS=1. |
|
| 37537 |
37 d 8 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
carlcare activation: retry a transient failure once before giving up
One retry after a 1.5s backoff - long enough to clear the far end's rate window,
short enough to still fit inside a scheduler tick. A transient network blip was
otherwise recorded as a hard activation failure for that IMEI. |
|
| 37518 |
39 d 20 h |
ranu |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
notification scheduler |
|
| 37510 |
40 d 13 h |
ranu |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
sales target and achievement added cron and flags |
|
| 37503 |
40 d 15 h |
ranu |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
sales target and achievement added cron and flags |
|
| 37499 |
40 d 15 h |
ranu |
/trunk/ |
sales target and achievement added cron and flags |
|
| 37492 |
40 d 19 h |
amit |
/trunk/ |
errors: wire the GlitchTip appender into cron and fofo logging
Completes r37491, which committed only web's log4j2.xml. cron's and fofo's were
held back because they carried local development log paths -- a user.home
expansion in cron and an absolute /Users path in fofo. Neither can ship: /Users
does not exist on the server, so fofo logging would fail to open its file and
the Alloy stream tailing /var/log/tomcat7/fofo/fofo.log would go dead.
Both files are restored to their production paths and carry the Sentry appender.
Whoever needs the local override should re-apply it locally rather than
committing it. |
|
| 37491 |
40 d 19 h |
amit |
/trunk/ |
errors: ship ERROR events to GlitchTip via the log4j2 Sentry appender
io.sentry:sentry-log4j2:7.22.6 in all three deployables. Pinned to 7.x because
8.x drops Java 8; verified class major version 52.
minimumEventLevel=ERROR is load-bearing. It only became safe after r37490 moved
business validations to WARN -- 32% of the ERROR stream was HTTP 400s where the
user is simply told what to fix, and sending those would have made "insufficient
balance" the top issue and buried real bugs. Breadcrumbs come from INFO so an
issue arrives with the log lines that preceded it.
Attached to the application loggers only; framework noise is not our bug.
web's log4j2.xml is committed here. cron's and fofo's are held back: both carry
uncommitted local development paths (a user.home expansion, and an absolute
/Users path) that would break production logging and the Alloy log tailing.
Their Sentry blocks are staged locally and should land with whoever owns those
path edits. |
|
| 37485 |
41 d 13 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
carlcare: enable tecno alongside itel
Itel went first because it is the brand we can CHECK -- the DCR pull independently
produces itel dates to reconcile against, where tecno has no second source at all.
That check has passed:
- a 50-imei read-only trial answered 50/50, no errors, no sign rejections, and
agreed with our catalog brand on all 50
- 18 of the 50 carried a date, and every one of those 18 fell AFTER our own
billing date (3 to 1,111 days, median ~220)
- the first live tick after deploy returned the same rate: 16 dates, 32 not-yet-
activated, 1 malformed response absorbed by the has("status") guard
Tecno is also the reason this class exists. Itel was already served by the DCR pull;
tecno has been served by nothing since 2024-11-20 and has been manual CSV ever since.
Capacity: ~10,400 itel + ~1,890 tecno pending against 14,400 lookups a day. The
round-robin splits a tick four ways only while all four queues have work; tecno is
much the smaller pool, so it is exhausted a few hours in, its queues then come back
empty and itel gets the full 50 again for the rest of the day. Itel still clears
daily -- roughly 1,875 + 10,650 = 12,525 itel lookups against a 10,401 pool. |
|
| 37484 |
41 d 13 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/monitored/ |
cron metrics: register the status gauge as -1, not 0
The gauge was created when a job STARTS, and 0 means failure, so any job still
running its first execution after a restart reported FAILED. Selenium-driven and
report jobs take minutes, so this is not a narrow window -- it put ten jobs on
the dashboard as failed after the 2026-08-31 restart when none had failed.
-1 means 'has not finished a run yet'. CronJobFailing matches == 0, so those
jobs are simply absent from the alert until they genuinely complete once. |
|
| 37482 |
41 d 14 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
carlcare: recover tecno/itel activation dates, and fix trunk broken by r37479
r37479/r37480 swept a working-copy edit to Application.java into the mail commit: trunk
has called scheduledTasks.checkCarlcareImeiActivation() since then without containing it,
so profitmandi-cron has not compiled. This adds the rest.
WHAT THIS IS
The tecno feed has been dead since 2024-11-20, when transsion decommissioned the SAP
OData hosts cms.tecno-mobile.com:8099 / cms.itel-mobile.com:8099. The imwav DCR portal
recovers itel but has nothing for tecno -- all three tecno logins together expose 17
imeis over five years, against 31,928 tecno rows in fofo.activated_imei. Everything since
has been manual CSV via /imei/upload.
Carlcare is transsion's own after-sales arm, and the warranty-check page on carlcare.in is
backed by a public per-imei endpoint that answers for tecno, itel and infinix with no
login, no cookie and no captcha:
GET ind-mis-carl.shalltry.com/CarlcareClient/electronic-card/check-extended_warranty-web?imei=
sign: md5(SALT + imei)
The sign header is mandatory (without it: code 10022 "Sorry, web sign is error"). The salt
is in the site's own bundle, _nuxt/085537c.js module 688, along with the base url; re-read
that bundle if it ever stops working. status 3 = activated and carries activeTime, status
2 = device known but not activated yet and activeTime is null.
It is also the semantically right source. The DCR portal serves an INVENTORY report and
the old SAP feed served a TERTIARY SALES report, whereas activeTime is the date the
handset was actually activated -- which is what fofo.activated_imei is meant to hold and
what tertiary payout is computed on.
Measured against hadb1 before writing any of this: tecno 355463920708766 -> 2026-08-28 and
itel 359207322028000 -> 2026-08-30, both exact matches to rows we already had. A read-only
trial of 50 itel imeis answered 50/50 with no errors and no sign rejections, agreed with
our catalog brand on all 50, and returned a date for 18 -- every one of those 18 falling
AFTER our own billing date, 3 to 1,111 days, median ~220.
SHAPE
Pool queries and saveActivation semantics are the vivo ones, so the two read alike, and no
DAO change was needed: the pending queries are already brand-generic. The far end is far
cheaper than vivo's, one signed GET per imei, so there is no captcha service, no cookie
store, no session seeding and no verdict reporting.
50 imeis every 5 minutes = 14,400 lookups a day against ~10,400 pending itel, so the whole
pool is covered daily with headroom. The pool reaches back to 2021, so the ticks are
themselves the backfill of the nov-2024 blackout; there is no one-off to run.
Two things worth knowing before changing it:
- The batch is drawn ROUND-ROBIN across each (brand, channel) queue, not by concatenating
them. A full daily pass can concatenate freely because it walks to the end, but a
50-at-a-time tick cannot: itel is ~1,550 secondary against ~8,851 tertiary, so the head
of a concatenated list is ~31 straight ticks of pure secondary before one tertiary imei
is asked about. The first trial batch was 100% secondary for exactly that reason.
- Every outcome stamps the row, failures included. This is the one deliberate departure
from vivo, which leaves a failure unrecorded so it retries next pass -- safe there
because the next pass is tomorrow. On a 5-minute cadence it is not: an unstamped imei is
due again in five minutes, the query keeps handing back the same 50 rows, the batch never
advances past them, and the endpoint is asked the same questions twelve times an hour for
as long as it keeps failing. See the runaway documented on oppoRealmeImeiActivation.
BRANDS is itel alone to start. That is a rollout order, not a limit of the endpoint: itel
is the brand we can CHECK, because the DCR pull independently produces itel dates to
reconcile against, where tecno has nothing. Add "Tecno" once a day's rows agree. Both feeds
may write itel meanwhile with no coordination -- the pool query only returns imeis whose
activationTimestamp is still null, so whatever one fills has left the other's pool.
Scheduled tick plus a --checkCarlcareImeiActivation flag, both wired. |
|
| 37480 |
41 d 14 h |
amit |
/trunk/ |
mail: correct a wrong claim in r37479 -- the Google app password IS valid
r37479 stated googleMailSender's app password was rejected. That was wrong. The
test behind it resolved smtp.gmail.com over IPv6; repeating it over IPv4 with
the same credential gives AUTH OK on both 465 and 587.
The real fault is not the credential and not the bean config, both of which are
correct. SMTP from this host works over IPv4 only:
smtp.gmail.com IPv4 -> AUTH OK IPv6 -> 535 5.7.8 Username and Password not accepted
smtp-relay IPv4 -> 250 MAIL FROM IPv6 -> 550 5.7.1 Invalid credentials for relay
The JVM prefers IPv4, which is the only reason mail leaves this box at all.
Anything that prefers IPv6 fails on both paths. |
|
| 37479 |
41 d 14 h |
amit |
/trunk/ |
mail: make the relay the default sender, not the Google identity
Correcting r37474-37476. Those made googleMailSender @Primary on the assumption
its credentials worked. Tested against the live servers from the prod host:
googleMailSender 535 5.7.8 Username and Password not accepted (BadCredentials)
-- the app password in the source is no longer valid
relay over IPv4 250 OK for MAIL FROM:<noreply@smartdukaan.com>
relay over IPv6 550 5.7.1 Invalid credentials for relay
So promoting google would have replaced one broken default with another. The
relay is what actually delivers today and it becomes 'mailSender'. It does not
authenticate -- Workspace authorises it by allowlisted source IP -- so sending
as noreply@ is legitimate there and AuthenticatedIdentityMailSender correctly
leaves it alone.
googleMailSender stays available by qualifier. Point the default back at it once
a valid app password is issued for sdtech@smartdukaan.com.
Also noted in the javadoc: the relay allowlist covers the IPv4 address only, so
anything that prefers IPv6 will be refused. |
|