| Rev |
Age |
Author |
Path |
Log message |
Diff |
| 37349 |
7 h 24 m |
ranu |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/dao/repository/transaction/ |
super retailer club 5 live |
|
| 37348 |
7 h 28 m |
ranu |
/trunk/ |
super retailer club 5 live |
|
| 37347 |
8 h 56 m |
amit |
/trunk/profitmandi-fofo/src/main/java/com/spice/profitmandi/web/offercircular/ |
Give the ingest runner a transaction; it had none on its background thread
Parsing failed instantly on production with
org.hibernate.HibernateException: Could not obtain transaction-synchronized
Session for current thread
at CircularIngestRepositoryImpl.claimForProcessing
at CircularIngestRunner.processOne
The runner works on a background thread. Nothing binds a Hibernate session to it, so
the very first repository call - the claim - threw. Worse, the failure handler called
markFailed, which threw for the same reason, so nothing was recorded: the document sat
in DRAFT with no error, no processed_at and no outward sign that anything had gone
wrong. CircularIngestService.ingest was never reached.
This was latent in the cron version too. It never surfaced because that scheduler was
never actually deployed anywhere.
- New CircularIngestBookkeeping: claim / document / published / failed /
reclaimStalled, each REQUIRES_NEW. Independent transactions matter most for failed(),
which runs after the ingest transaction has already rolled back and must not be
dragged into it.
- It is a SEPARATE bean on purpose. @Transactional on the runner's own methods would be
invoked from inside its own Runnable - a self-invocation never passes through the
Spring proxy, so the annotation would be silently ignored and the bug would come back
wearing a disguise.
- The runner no longer touches CircularIngestRepository at all.
Not caught locally because the RunIngest harness uses JdbcIngestRepo, a plain-JDBC
implementation that bypasses Hibernate entirely - so it can reproduce the parsing but
never a session or transaction problem. |
|
| 37346 |
9 h 9 m |
amit |
/trunk/profitmandi-fofo/src/main/java/com/spice/profitmandi/web/controller/ |
Sort sale returns so the ones still needing an action come first
The Recently Received Debit Notes table was ordered purely by receive time, so a
return waiting on a refund or on the partner's acknowledgment sank below rows that
were already refunded and needed nothing. On a busy warehouse the actionable rows
were off the first screen.
- recentOrders is now ranked pending-first (Received - Pending Refund, Rejected -
Pending Acknowledgment) ahead of settled ones (Refunded, Cancelled), with the
existing receiveTimestamp DESC kept as the tie-breaker inside each group
- getPendingActionRank follows the same precedence invoice-return.vm uses to pick
the status badge - reject checked before refund - so the ordering can never
disagree with the label the user sees
- ranked on the return's own state, not on the viewer's canReceive/canRefund
permissions, so a pending return stays at the top for everyone looking at it
Presentation only; no query, entity or lifecycle change. Note that returns never
received at all cannot surface here regardless, since the query filters on
receiveTimestamp BETWEEN the selected dates. |
|
| 37345 |
9 h 43 m |
vikas |
/trunk/profitmandi-fofo/src/main/java/com/spice/profitmandi/web/controller/ |
Beat Journey (today): LOI-based 'Onboarded' + last-billing recency board columns, level-filtered orders list (grouped by partner), and flag tuning — remove #1/#3, use per-visit total_distance for #4.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|
| 37344 |
9 h 50 m |
vikas |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/dao/repository/user/ |
Beat Journey: LoiFormRepository.selectMaxApprovedAcquiredDateByBdm() — max approved-LOI acquiredDate per BDM, powering the board 'Onboarded' column (store.active_timestamp is null for all stores).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|
| 37343 |
10 h 36 m |
vikas |
/trunk/profitmandi-fofo/src/main/ |
Update PJP, Visit quality flags |
|
| 37342 |
10 h 55 m |
amit |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/dao/repository/offers/ |
Allow re-ingest to start a circular already sitting in DRAFT
Moving the parse into the portal turned DRAFT into a dead end. Under the cron
scheduler DRAFT meant "something will pick me up", so re-ingest only ever had to push
a document back into it, and PUBLISHED/FAILED were the only sensible sources. Nothing
polls DRAFT any more - the runner is triggered by the upload that created the
document - so a circular whose trigger never fired had no way to be started at all.
That is not hypothetical: the Aug'26 circular uploaded to production before the runner
was deployed sits in DRAFT, and pressing re-ingest matched zero rows and reported
"not started".
DRAFT now joins PUBLISHED and FAILED as a valid source state. PROCESSING stays
excluded - that one is genuinely in flight - and claimForProcessing remains the single
arbiter of who actually parses, so allowing DRAFT cannot cause a double parse. |
|
| 37341 |
11 h 2 m |
amit |
/trunk/profitmandi-web/src/main/java/com/spice/profitmandi/web/v2/controller/fofo/ |
fix(rbm-dashboard): return full partner book and exclude inactive stores
/v2/fofo/rbm-dashboard capped partners/alerts at 50 by default. The RBM home
searches and filters client-side, so any partner outside that slice was
unfindable while summary.totalPartners reported the real (much larger) count.
- limit now defaults to 0 = uncapped; an explicit ?limit=n is still honoured
up to RBM_DASHBOARD_MAX_LIMIT (200)
- deactivated stores are skipped for the whole computation, not just alerts:
they have no target, scored 0% and sorted to the top of the action list
- summary.limit reports 0 when uncapped instead of Integer.MAX_VALUE |
|
| 37340 |
11 h 9 m |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Fixed max limit to 15 lac for Credit limit |
|
| 37339 |
12 h 6 m |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Fixed max limit to 15 lac for Credit limit |
|
| 37338 |
12 h 17 m |
amit |
/trunk/profitmandi-cron/ |
Remove offer-circular ingest from cron - it now runs in the portal
Counterpart to the fofo commit that moved the parse into profitmandi-fofo. Nothing is
lost: all 13 parser classes and the test moved verbatim, and CircularIngestScheduler
was replaced by an executor-driven runner in the portal.
- com.smartdukaan.cron.offercircular deleted, main and test
- tabula dependency removed; PDFBox no longer enters this artifact at all
- dumpCircularClasspath moved to fofo, where the parser and its classpath now live
This module no longer knows anything about offer circulars, so a cron rebuild is no
longer a prerequisite for the feature to work - which was the entire problem. |
|
| 37337 |
12 h 17 m |
amit |
/trunk/profitmandi-fofo/ |
Move offer-circular ingest out of cron and into the portal
The parse now runs in profitmandi-fofo, on a background thread, triggered by the
upload that produced the document.
Why: splitting one feature across two artifacts with independent deploy cadences
cost a full day. fofo shipped, cron did not, and a valid upload sat in DRAFT with
nothing on the server able to parse it - the deployed cron jar contained none of the
ingest classes. One 9-page PDF a month never justified a batch tier, and the portal
already ships two PDF stacks, so the isolation argument for keeping PDFBox out was
weaker than it looked.
- 13 parser classes move verbatim from com.smartdukaan.cron.offercircular to
com.spice.profitmandi.web.offercircular. No logic changed.
- CircularIngestScheduler becomes CircularIngestRunner: the @Scheduled(every 5 min)
entry point and the offer.circular.ingest.enabled flag are gone, replaced by a
single-threaded daemon executor. All claim, ingest and notification logic is
unchanged.
- Upload hands the document id to the runner AFTER COMMIT, not inline. The DRAFT row
is written inside the request transaction; a worker starting immediately would race
that commit, find nothing to claim and silently do nothing - which is precisely the
stuck-on-DRAFT symptom this change removes.
- The guarded claim is KEPT even though there is now one trigger. It still stops a
double-submit, a second portal node, and a re-ingest racing an in-flight parse.
- Re-ingest parses immediately instead of queueing for a scheduler.
- The stall reaper runs when the review screen loads. There is no timer here any
more, and a document stranded by a redeploy mid-parse only matters when somebody
looks for it - which matters more now the parse lives in the web application.
- tabula moves to this module with its exclusions intact, as does the
dumpCircularClasspath helper the local ingest harness depends on.
- Screen no longer claims "the ingest job runs every 5 minutes", which was untrue the
moment cron stopped being the route; poll interval 15s -> 3s to match a parse that
takes seconds. jsVersion 404 -> 405.
- offer.circular.review.url added here, since the runner sends that email now.
Verified: full ingest of the Aug'26 circular through the relocated code is identical
to the reference - 239 offers, 663 products, 279 AUTO_EXACT, 361 benefits, 511
tenures, 911 bank links. ProductNamesTest 11/11 in its new home. |
|
| 37336 |
13 h 41 m |
amit |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/service/ |
Expose loginEmail to the admin views
admin.vm gates the OFFER CIRCULAR sidebar entry on the exact logged-in email,
because visibility there is per-email and auth.menu can only express roles. Nothing
was setting that variable, so the #if was always false and the entry rendered for
nobody on a clean build - it only appeared where a working copy happened to carry
this line locally.
Committed alongside r37331, which added the gate itself. |
|
| 37335 |
13 h 43 m |
amit |
/trunk/profitmandi-fofo/src/main/ |
Show offer circular ingest state on the review screen
Upload and parsing run in different processes - the portal stores the PDF as DRAFT
and the cron job claims it minutes later - so the screen looked identical before and
after a successful ingest. A reviewer had no way to tell "not parsed yet" from
"parsed and produced nothing", and a FAILED circular showed nothing at all.
- the selected circular's state is shown as queued / parsing / parsed / failed,
with the recorded reason on failure and the ingest summary on success
- while it is DRAFT or PROCESSING the screen polls every 15s and pulls in the rows
the moment the parse lands. Polling stops as soon as the state settles, so it
cannot spin forever, and a transient error keeps waiting rather than giving up
- jsVersion 403 -> 404, since offer-circular-review.js is already being served and
this is a modification rather than a new file
Pairs with the stall reaper: a circular abandoned mid-parse is now both recovered
and explained on screen instead of sitting silently in PROCESSING. |
|
| 37334 |
13 h 43 m |
amit |
/trunk/profitmandi-cron/ |
Sweep stalled offer circulars before each ingest run, and drop stale scaffolding
- The scheduler now calls reclaimStalledProcessing before looking for new work, so
a circular stranded in PROCESSING by a dead or redeployed worker is failed with a
reason instead of staying invisible forever. Threshold is 30 minutes against a
parse that takes seconds, so it can only ever catch a genuinely dead worker. The
sweep is wrapped so it can never stop the actual ingest.
- dumpCircularClasspath was labelled a temporary helper for diffing against the
reference Python. That Python has been deleted, but the task is what lets an
ingest be reproduced and measured locally per scripts/offer_circular/README.md,
so the comment now says what it is for rather than telling the next reader to
delete it.
- CircularExtractor's javadoc claimed verification against the Python reference.
That proved equivalence, not correctness - both shared the missing-memory-unit
bug that bound offers to the wrong SKU. Reworded so it cannot be read as a
correctness guarantee, and points at the fixture and ProductNamesTest instead.
Verified: full ingest of the Aug'26 circular is byte-identical to the reference -
239 offers, 663 products, 279 AUTO_EXACT. ProductNamesTest 11/11. |
|
| 37333 |
13 h 43 m |
amit |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/dao/repository/offers/ |
Reclaim offer circulars stranded in PROCESSING, and surface the ingest reason
A claim moves a circular DRAFT -> PROCESSING, and selectDraftDocumentIds only ever
picks up DRAFT. So a worker that died, was redeployed, or hung mid-parse left that
document stranded in PROCESSING forever, invisible to every later sweep. The batch
tier looked like it provided retry semantics and did not.
- reclaimStalledProcessing(minutes) fails anything left in PROCESSING past the
threshold. Deliberately FAILED and not back to DRAFT: a circular that reliably
kills the parser would otherwise be re-claimed on every sweep forever. FAILED is
visible, carries the reason, and the existing re-ingest action already allows
FAILED -> DRAFT, so re-queueing is one click.
- claimForProcessing now stamps processed_at, which is what the stall is measured
against. No DDL: that column is written only by this repository and read by
nothing else, so it can carry the last-transition time.
- selectDocuments returns ingest_error and ingest_summary. Without the reason the
screen would show a bare FAILED, which the reaper would now produce routinely.
Verified against the local DB both ways: a claim aged 45 minutes is reclaimed with
its reason recorded, a fresh claim is untouched. |
|
| 37332 |
14 h 36 m |
amit |
/trunk/profitmandi-fofo/src/main/ |
Move the offer-circular upload directory off a developer home path
Every upload in production failed with a 400 on mkdirs. offer.circular.dir was
never set in any environment, so it fell back to the code default
/users/amit/uploads/offer-circulars; Tomcat runs as the tomcat user, which cannot
create /users at the filesystem root, and the controller threw "Circular storage
directory is not writable".
- default is now /var/lib/smartdukaan/offer-circulars, a real server path
- offer.circular.dir set explicitly in prod.properties, and pointed at the
developer's own space in dev.properties, since /var/lib needs sudo on a
workstation
- staging.properties deliberately untouched; the new default covers it
The location has two hard constraints, now recorded on the field itself:
- OUTSIDE webapps/. A redeploy wipes anything under it, and
offer_document.stored_path is how both re-ingest and re-download resolve the
PDF, so a wipe breaks them permanently. Stray PDFs are already sitting in
WEB-INF/classes/META-INF on the server from exactly this mistake.
- OUTSIDE any web root. A circular carries the OEM's confidential cashback
economics and must never be servable as a static file. Hence /var/lib rather
than /var/www, whose local precedent /var/www/partner_stats is chmod 777.
Only fofo needs the key: the cron ingest resolves the PDF from
offer_document.stored_path and reads it as root, so 0750 tomcat:tomcat is enough.
Deploying this alone does not fix production - the directory must also exist and
be writable by tomcat. |
|
| 37331 |
1 d 1 h |
amit |
/trunk/profitmandi-fofo/src/main/ |
Add offer circular review and curation screen
Shows the nine verbatim PDF columns beside what the parser made of them, uploads a
new monthly circular, and resolves the products the matcher could not.
Upload deliberately does not parse - it stores the PDF, sha256-deduped, and
registers it DRAFT, so an upload can never half-populate the offer tables. The cron
ingest job picks it up from there.
The curation queue groups by distinct (division, raw text) because that is the
product_alias key: one answer clears every row carrying that text, on this circular
and on every later one. On the Aug'26 circular that turns 152 unresolved rows into
106 decisions. Three actions mirror product_alias.action - PIN to a catalog id,
REWRITE to the catalog's spelling and match again, or IGNORE.
- Writes product_alias only, so nothing the screen does can set manually_curated
and lock a circular against re-ingest. Coverage decisions, which do need that,
are listed but visibly parked rather than offered a naming answer.
- A saved decision changes nothing until re-ingest, because aliases are read at
ingest time; the queue says so and the re-ingest button puts the document back
to DRAFT for the scheduler. The portal cannot run the ingest itself - that lives
in profitmandi-cron, which fofo does not depend on.
- The closest rejected match is shown for context and never pre-selected. Four new
OnePlus models each "nearly" matched a 2015 OnePlus 2 at 0.80-0.84, and offering
that as a suggestion invites a reviewer to confirm it. Where the catalog has no
such SKU the search says so and steers to IGNORE.
- aliasExport emits every decision as replayable SQL, because the portal writes to
one database and environments would otherwise drift apart silently.
Access is an email allowlist AND the admin role. Both are enforced in the
controller; the sidebar entry repeats the allowlist because visibility in auth.menu
is role-driven and cannot express a per-email rule.
No jsVersion bump: offer-circular-review.js is a new file that was never cached,
so it needs no cache-buster, and churning that shared counter only conflicts with
whoever is mid-edit on it. |
|
| 37330 |
1 d 4 h |
amit |
/trunk/profitmandi-cron/ |
Add Pine Labs affordability circular parsing and ingest
Turns the monthly OEM "Mobile & Laptop Offers" PDF into the offers schema: tabula
extraction to nine verbatim columns per row, per-cell parsers for benefit, tenure,
bank and footnote text, catalog matching, and a scheduler that claims DRAFT
documents uploaded from the FOFO portal and emails the uploader the outcome.
Ships inert. offer.circular.ingest.enabled defaults to false, so the scheduler does
nothing until an environment opts in, and offer.circular.review.url defaults to
empty - neither key is required for the context to start.
- tabula added here and not in profitmandi-common so PDFBox never reaches the
web/fofo WARs; bouncycastle, slf4j-simple and jai-imageio excluded (version
clash, duplicate SLF4J binding, and unused image decoding respectively)
- offer_raw_row holds all nine columns verbatim, so re-parsing reads the table and
never the PDF again
- product_alias is consulted before matching, so a human confirmation recorded once
keeps applying every following month
ProductNamesTest locks in the product-name parsing, which decides which SKU an
offer's money lands on. Every case there is a real mis-parse, and they share one
root cause: characters or digits belonging to the model name being eaten as memory
or stripped as punctuation. Two worth naming:
- A memory spec with no GB/TB unit is still a memory spec. Motorola writes
"(8+256)" and vivo "(8+256G)"; unrecognised, the matcher believed no size was
given and bound the offer to an arbitrary sibling - a 1,000 Edge 60 Pro 8+256
offer and a 2,000 12+256 offer landed on the same SKU.
- Two variant groups written back-to-back are two products. "Edge 70 Pro
(8+256)(12+256)" stayed one product bound to a single SKU while the 12+256
variant silently got no offer at all.
Bundled accessories are deliberately NOT stripped back to the bare phone. Doing so
resolves 30 CATALOG_GAP rows and looks safe on Oppo, whose bundled and bare rows
carry identical values - but vivo caps X300 Pro(16+512G) at 10,000 on its own row
and 11,000 on the "+Extender" row, the difference being the Extender. Merging them
would let the bundle's cap be claimed on a phone sold without the accessory.
Whether a bundle offer transfers to the bare SKU is a commercial question the PDF
does not answer, so it is a manual coverage decision, not a parsing rule.
Verified against the Aug'26 circular on the local DB: 324 rows in the PDF, 239
loaded, producing 361 benefits, 511 tenures, 911 bank links and 663 products, of
which 74% resolve automatically. ProductNamesTest 11/11. |
|