| Rev |
Age |
Author |
Path |
Log message |
Diff |
| 37339 |
28 m |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Fixed max limit to 15 lac for Credit limit |
|
| 37338 |
39 m |
amit |
/trunk/profitmandi-cron/ |
Remove offer-circular ingest from cron - it now runs in the portal
Counterpart to the fofo commit that moved the parse into profitmandi-fofo. Nothing is
lost: all 13 parser classes and the test moved verbatim, and CircularIngestScheduler
was replaced by an executor-driven runner in the portal.
- com.smartdukaan.cron.offercircular deleted, main and test
- tabula dependency removed; PDFBox no longer enters this artifact at all
- dumpCircularClasspath moved to fofo, where the parser and its classpath now live
This module no longer knows anything about offer circulars, so a cron rebuild is no
longer a prerequisite for the feature to work - which was the entire problem. |
|
| 37337 |
39 m |
amit |
/trunk/profitmandi-fofo/ |
Move offer-circular ingest out of cron and into the portal
The parse now runs in profitmandi-fofo, on a background thread, triggered by the
upload that produced the document.
Why: splitting one feature across two artifacts with independent deploy cadences
cost a full day. fofo shipped, cron did not, and a valid upload sat in DRAFT with
nothing on the server able to parse it - the deployed cron jar contained none of the
ingest classes. One 9-page PDF a month never justified a batch tier, and the portal
already ships two PDF stacks, so the isolation argument for keeping PDFBox out was
weaker than it looked.
- 13 parser classes move verbatim from com.smartdukaan.cron.offercircular to
com.spice.profitmandi.web.offercircular. No logic changed.
- CircularIngestScheduler becomes CircularIngestRunner: the @Scheduled(every 5 min)
entry point and the offer.circular.ingest.enabled flag are gone, replaced by a
single-threaded daemon executor. All claim, ingest and notification logic is
unchanged.
- Upload hands the document id to the runner AFTER COMMIT, not inline. The DRAFT row
is written inside the request transaction; a worker starting immediately would race
that commit, find nothing to claim and silently do nothing - which is precisely the
stuck-on-DRAFT symptom this change removes.
- The guarded claim is KEPT even though there is now one trigger. It still stops a
double-submit, a second portal node, and a re-ingest racing an in-flight parse.
- Re-ingest parses immediately instead of queueing for a scheduler.
- The stall reaper runs when the review screen loads. There is no timer here any
more, and a document stranded by a redeploy mid-parse only matters when somebody
looks for it - which matters more now the parse lives in the web application.
- tabula moves to this module with its exclusions intact, as does the
dumpCircularClasspath helper the local ingest harness depends on.
- Screen no longer claims "the ingest job runs every 5 minutes", which was untrue the
moment cron stopped being the route; poll interval 15s -> 3s to match a parse that
takes seconds. jsVersion 404 -> 405.
- offer.circular.review.url added here, since the runner sends that email now.
Verified: full ingest of the Aug'26 circular through the relocated code is identical
to the reference - 239 offers, 663 products, 279 AUTO_EXACT, 361 benefits, 511
tenures, 911 bank links. ProductNamesTest 11/11 in its new home. |
|
| 37336 |
2 h 3 m |
amit |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/service/ |
Expose loginEmail to the admin views
admin.vm gates the OFFER CIRCULAR sidebar entry on the exact logged-in email,
because visibility there is per-email and auth.menu can only express roles. Nothing
was setting that variable, so the #if was always false and the entry rendered for
nobody on a clean build - it only appeared where a working copy happened to carry
this line locally.
Committed alongside r37331, which added the gate itself. |
|
| 37335 |
2 h 5 m |
amit |
/trunk/profitmandi-fofo/src/main/ |
Show offer circular ingest state on the review screen
Upload and parsing run in different processes - the portal stores the PDF as DRAFT
and the cron job claims it minutes later - so the screen looked identical before and
after a successful ingest. A reviewer had no way to tell "not parsed yet" from
"parsed and produced nothing", and a FAILED circular showed nothing at all.
- the selected circular's state is shown as queued / parsing / parsed / failed,
with the recorded reason on failure and the ingest summary on success
- while it is DRAFT or PROCESSING the screen polls every 15s and pulls in the rows
the moment the parse lands. Polling stops as soon as the state settles, so it
cannot spin forever, and a transient error keeps waiting rather than giving up
- jsVersion 403 -> 404, since offer-circular-review.js is already being served and
this is a modification rather than a new file
Pairs with the stall reaper: a circular abandoned mid-parse is now both recovered
and explained on screen instead of sitting silently in PROCESSING. |
|
| 37334 |
2 h 5 m |
amit |
/trunk/profitmandi-cron/ |
Sweep stalled offer circulars before each ingest run, and drop stale scaffolding
- The scheduler now calls reclaimStalledProcessing before looking for new work, so
a circular stranded in PROCESSING by a dead or redeployed worker is failed with a
reason instead of staying invisible forever. Threshold is 30 minutes against a
parse that takes seconds, so it can only ever catch a genuinely dead worker. The
sweep is wrapped so it can never stop the actual ingest.
- dumpCircularClasspath was labelled a temporary helper for diffing against the
reference Python. That Python has been deleted, but the task is what lets an
ingest be reproduced and measured locally per scripts/offer_circular/README.md,
so the comment now says what it is for rather than telling the next reader to
delete it.
- CircularExtractor's javadoc claimed verification against the Python reference.
That proved equivalence, not correctness - both shared the missing-memory-unit
bug that bound offers to the wrong SKU. Reworded so it cannot be read as a
correctness guarantee, and points at the fixture and ProductNamesTest instead.
Verified: full ingest of the Aug'26 circular is byte-identical to the reference -
239 offers, 663 products, 279 AUTO_EXACT. ProductNamesTest 11/11. |
|
| 37333 |
2 h 5 m |
amit |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/dao/repository/offers/ |
Reclaim offer circulars stranded in PROCESSING, and surface the ingest reason
A claim moves a circular DRAFT -> PROCESSING, and selectDraftDocumentIds only ever
picks up DRAFT. So a worker that died, was redeployed, or hung mid-parse left that
document stranded in PROCESSING forever, invisible to every later sweep. The batch
tier looked like it provided retry semantics and did not.
- reclaimStalledProcessing(minutes) fails anything left in PROCESSING past the
threshold. Deliberately FAILED and not back to DRAFT: a circular that reliably
kills the parser would otherwise be re-claimed on every sweep forever. FAILED is
visible, carries the reason, and the existing re-ingest action already allows
FAILED -> DRAFT, so re-queueing is one click.
- claimForProcessing now stamps processed_at, which is what the stall is measured
against. No DDL: that column is written only by this repository and read by
nothing else, so it can carry the last-transition time.
- selectDocuments returns ingest_error and ingest_summary. Without the reason the
screen would show a bare FAILED, which the reaper would now produce routinely.
Verified against the local DB both ways: a claim aged 45 minutes is reclaimed with
its reason recorded, a fresh claim is untouched. |
|
| 37332 |
2 h 58 m |
amit |
/trunk/profitmandi-fofo/src/main/ |
Move the offer-circular upload directory off a developer home path
Every upload in production failed with a 400 on mkdirs. offer.circular.dir was
never set in any environment, so it fell back to the code default
/users/amit/uploads/offer-circulars; Tomcat runs as the tomcat user, which cannot
create /users at the filesystem root, and the controller threw "Circular storage
directory is not writable".
- default is now /var/lib/smartdukaan/offer-circulars, a real server path
- offer.circular.dir set explicitly in prod.properties, and pointed at the
developer's own space in dev.properties, since /var/lib needs sudo on a
workstation
- staging.properties deliberately untouched; the new default covers it
The location has two hard constraints, now recorded on the field itself:
- OUTSIDE webapps/. A redeploy wipes anything under it, and
offer_document.stored_path is how both re-ingest and re-download resolve the
PDF, so a wipe breaks them permanently. Stray PDFs are already sitting in
WEB-INF/classes/META-INF on the server from exactly this mistake.
- OUTSIDE any web root. A circular carries the OEM's confidential cashback
economics and must never be servable as a static file. Hence /var/lib rather
than /var/www, whose local precedent /var/www/partner_stats is chmod 777.
Only fofo needs the key: the cron ingest resolves the PDF from
offer_document.stored_path and reads it as root, so 0750 tomcat:tomcat is enough.
Deploying this alone does not fix production - the directory must also exist and
be writable by tomcat. |
|
| 37331 |
13 h 58 m |
amit |
/trunk/profitmandi-fofo/src/main/ |
Add offer circular review and curation screen
Shows the nine verbatim PDF columns beside what the parser made of them, uploads a
new monthly circular, and resolves the products the matcher could not.
Upload deliberately does not parse - it stores the PDF, sha256-deduped, and
registers it DRAFT, so an upload can never half-populate the offer tables. The cron
ingest job picks it up from there.
The curation queue groups by distinct (division, raw text) because that is the
product_alias key: one answer clears every row carrying that text, on this circular
and on every later one. On the Aug'26 circular that turns 152 unresolved rows into
106 decisions. Three actions mirror product_alias.action - PIN to a catalog id,
REWRITE to the catalog's spelling and match again, or IGNORE.
- Writes product_alias only, so nothing the screen does can set manually_curated
and lock a circular against re-ingest. Coverage decisions, which do need that,
are listed but visibly parked rather than offered a naming answer.
- A saved decision changes nothing until re-ingest, because aliases are read at
ingest time; the queue says so and the re-ingest button puts the document back
to DRAFT for the scheduler. The portal cannot run the ingest itself - that lives
in profitmandi-cron, which fofo does not depend on.
- The closest rejected match is shown for context and never pre-selected. Four new
OnePlus models each "nearly" matched a 2015 OnePlus 2 at 0.80-0.84, and offering
that as a suggestion invites a reviewer to confirm it. Where the catalog has no
such SKU the search says so and steers to IGNORE.
- aliasExport emits every decision as replayable SQL, because the portal writes to
one database and environments would otherwise drift apart silently.
Access is an email allowlist AND the admin role. Both are enforced in the
controller; the sidebar entry repeats the allowlist because visibility in auth.menu
is role-driven and cannot express a per-email rule.
No jsVersion bump: offer-circular-review.js is a new file that was never cached,
so it needs no cache-buster, and churning that shared counter only conflicts with
whoever is mid-edit on it. |
|
| 37330 |
17 h 9 m |
amit |
/trunk/profitmandi-cron/ |
Add Pine Labs affordability circular parsing and ingest
Turns the monthly OEM "Mobile & Laptop Offers" PDF into the offers schema: tabula
extraction to nine verbatim columns per row, per-cell parsers for benefit, tenure,
bank and footnote text, catalog matching, and a scheduler that claims DRAFT
documents uploaded from the FOFO portal and emails the uploader the outcome.
Ships inert. offer.circular.ingest.enabled defaults to false, so the scheduler does
nothing until an environment opts in, and offer.circular.review.url defaults to
empty - neither key is required for the context to start.
- tabula added here and not in profitmandi-common so PDFBox never reaches the
web/fofo WARs; bouncycastle, slf4j-simple and jai-imageio excluded (version
clash, duplicate SLF4J binding, and unused image decoding respectively)
- offer_raw_row holds all nine columns verbatim, so re-parsing reads the table and
never the PDF again
- product_alias is consulted before matching, so a human confirmation recorded once
keeps applying every following month
ProductNamesTest locks in the product-name parsing, which decides which SKU an
offer's money lands on. Every case there is a real mis-parse, and they share one
root cause: characters or digits belonging to the model name being eaten as memory
or stripped as punctuation. Two worth naming:
- A memory spec with no GB/TB unit is still a memory spec. Motorola writes
"(8+256)" and vivo "(8+256G)"; unrecognised, the matcher believed no size was
given and bound the offer to an arbitrary sibling - a 1,000 Edge 60 Pro 8+256
offer and a 2,000 12+256 offer landed on the same SKU.
- Two variant groups written back-to-back are two products. "Edge 70 Pro
(8+256)(12+256)" stayed one product bound to a single SKU while the 12+256
variant silently got no offer at all.
Bundled accessories are deliberately NOT stripped back to the bare phone. Doing so
resolves 30 CATALOG_GAP rows and looks safe on Oppo, whose bundled and bare rows
carry identical values - but vivo caps X300 Pro(16+512G) at 10,000 on its own row
and 11,000 on the "+Extender" row, the difference being the Extender. Merging them
would let the bundle's cap be claimed on a phone sold without the accessory.
Whether a bundle offer transfers to the bare SKU is a commercial question the PDF
does not answer, so it is a manual coverage decision, not a parsing rule.
Verified against the Aug'26 circular on the local DB: 324 rows in the PDF, 239
loaded, producing 361 benefits, 511 tenures, 911 bank links and 663 products, of
which 74% resolve automatically. ProductNamesTest 11/11. |
|
| 37329 |
17 h 10 m |
amit |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/dao/repository/offers/ |
Add offers-schema repositories for the Pine Labs affordability circular
Three repositories over the offers schema, split by who is allowed to write what:
- OfferCircularRepository - read side for the review screen (verbatim rows,
documents, per-offer parsed detail)
- CircularIngestRepository - write side for ingest; claims a DRAFT document with
a guarded UPDATE and deletes only rows derived from that one document, so
re-ingesting one month cannot disturb another
- OfferCurationRepository - write side for manual curation
The split is deliberate: a caller that only displays circulars should not hold a
handle that can rewrite the scope config or the offer tables.
OfferCurationRepository writes offers.product_alias and nothing else. An alias is
config, keyed on (division_id, raw_text), so re-ingest reproduces it and one
decision carries into every later circular. Writing offer_product instead would be
derived data that re-ingest deletes, which is why doing so has to set
manually_curated and freeze the circular permanently - not a trade worth making for
a naming decision, so this repository cannot make it.
A PIN is validated against the division's own brand and category before it is
stored. Nothing downstream re-checks a pinned id - the matcher takes it at face
value - so an unverified exception would silently mismap money. |
|
| 37328 |
18 h 51 m |
amit |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/service/integrations/gstpro/ |
Fetch a fresh EWB auth token when the cached one is stale
Two defects made an expired EWB token unrecoverable, surfacing as GSP102
"eInvoice AuthToken not found or expired" on every GET.
1. ewbApiGet's retry was unreachable. NIC returns GSP102 with HTTP 400, and
RestClient.execute throws GE_1005 on any non-2xx before returning the body, so the
'if (response.contains("GSP102"))' branch could never run. A failed first attempt is
now treated as a possibly-stale token and retried once with a freshly minted one; a
second failure propagates. The POST path was unaffected — executeJson does not check
status, so its retry already worked.
2. Eviction targeted the wrong cache. The token is cached in redisCacheManager but all
three sites evicted through redisFortnightlyCacheManage, a different cache, so the
eviction was a silent no-op and the stale token survived. Eviction now goes through
GstProAuthService.evictEwbAuthToken, declared beside the @Cacheable and pinned to the
same cacheManager so the two cannot drift apart again.
Found while dry-running the EWB backfill, which failed on the first invoice. |
|
| 37327 |
20 h 31 m |
ranu |
/trunk/ |
super retailer club 5 live |
|
| 37326 |
20 h 53 m |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Add --backfillMissingEwb cron arg
Runs InvoiceService.backfillMissingEwb over an explicit invoice list:
--backfillMissingEwb --invoiceNumbers=A,B,C [--dryRun]
Explicit list rather than a date window so the backfill only touches the invoices known
to have lost their EWB. NOT_SUPPORTED on the RunOnceTasks wrapper suspends the
class-level transaction, keeping the NIC lookups outside one. |
|
| 37325 |
20 h 53 m |
amit |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/service/transaction/invoicing/ |
Add EWB backfill for invoices whose IRN was recovered via DUPIRN
When GenerateIRN carries EwbDtls, NIC files the IRN and e-way bill together and reports
the EWB back only in that response, flagged by InfoDtls[InfCd=EWBPPD]. A read timeout
sends the retry down the DUPIRN path, which recovers the IRN via the get-e-invoice
endpoint — that response carries neither InfoDtls nor EWB fields. The EWB therefore
exists at NIC while ewb_no stays null locally, and since updatePdfModelWithIrnDetails
gates the entire e-way bill section on ewb_no, the stored PDF omits it altogether.
backfillMissingEwb() looks the EWB up by (seller GSTIN, ack date), matches on docNo, and
takes actualDist from the per-EWB call since the distance is printed on the invoice. It
then re-renders the PDF; generateIrnForInvoices returns early once irn_generated is set,
so nothing is re-filed at NIC.
NIC is queried with the transaction suspended (NOT_SUPPORTED); the write and the
re-render each run in their own REQUIRES_NEW transaction. |
|
| 37324 |
21 h 44 m |
aman |
/trunk/profitmandi-fofo/src/main/java/com/spice/profitmandi/web/controller/ |
Lead Management: restrict lead view to field-level sales only (L1-L3). Any SALES position at any level previously forced the own+reportees lead filter, so senior sales heads (L4+ BM/RSM/NSM/co-founder L7) lost visibility of all leads and their dashboard charts collapsed. hasCategory check replaced with isFieldSales() at both view-filter sites; assignee validation untouched. |
|
| 37323 |
21 h 48 m |
amit |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/ |
Drop irn_attempt_count; IRN transport retry is unbounded
Removes the attempt counter added in r37322 along with its pending ALTER TABLE, so the
change no longer carries a schema dependency.
NIC outages resolve within the day, and an invoice legally requires an IRN, so capping
the retry would not remove the obligation — it would only stop trying. Transport failures
now stay queued (irn_generated NULL) until the provider recovers; only a genuine rejection
from NIC is terminal. The failure reason is still recorded in irn_error_message, which
distinguishes a requeued transport failure from an invoice never yet attempted. |
|
| 37322 |
22 h 31 m |
amit |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/ |
Stop treating IRN transport timeouts as terminal; isolate NIC calls from batch transaction
A read timeout to GSTPro/NIC was recorded as a final verdict (irn_generated=false),
so 79 invoices billed on 2026-08-17 were left permanently without an IRN even though
NIC may well have filed them. A timeout means the call never completed, not that the
document was rejected.
- markEInvoiceFailed -> recordIrnFailure(invoiceNumber, Throwable): transport failures
leave irn_generated NULL so the cron retries (DUPIRN recovers anything NIC did file);
only a genuine rejection is terminal. Alert email now fires only when terminal.
- New einvoice_details.irn_attempt_count bounds that retry at 10 attempts, reset on
success, so a prolonged NIC outage still converges instead of looping forever.
Requires the matching ALTER TABLE before deploy.
- New saveInvoiceInNewTransaction(invoiceNumber): REQUIRES_NEW per invoice, reloading
orders inside it. RunOnceTasks has class-level @Transactional wrapping the whole
batch loop, so every NIC call previously ran inside one transaction holding write
locks on all orders in the batch; at 60s per call that window is unacceptable.
updateIrnsToInvoices and regenerateBilledInvoices now carry only invoice numbers,
keeping the batch transaction read-only.
- Route all NIC calls (IRN gen, auth, cancel, EWB) through the 60s regulator profile
via GstProAuthService.nicRestClient(). getGstDetails stays on the 10s default since
it runs on request threads. |
|
| 37321 |
22 h 32 m |
amit |
/trunk/profitmandi-common/src/main/java/com/spice/profitmandi/common/web/client/ |
Preserve transport-failure cause in RestClient; add 60s regulator timeout profile
RestClient wrapped IOException/ClientProtocolException into RuntimeException(GE_1006)
without a cause, so callers could not tell a read timeout apart from a business
rejection. Pass the original exception as the cause at all four transport catch sites;
message text is unchanged.
Add HttpClientFactory.slowRegulatorRequestConfig() (60s socket) for NIC e-invoice/EWB
calls, which routinely exceed the global 10s default at peak, plus a RestClient
constructor taking an explicit RequestConfig. The global default is unchanged. |
|
| 37320 |
3 d 19 h |
ranu |
/trunk/profitmandi-dao/src/main/resources/ |
one assist ew at 99 up to 20k |
|