| Rev |
Age |
Author |
Path |
Log message |
Diff |
| 37678 |
1 m |
amit |
/trunk/profitmandi-fofo/src/main/java/com/spice/profitmandi/web/service/ |
Replace the knowlarity insights chrome scrape with SR's own JSON API
Drops the last unattended headless chrome out of the fofo tomcat. The scheduled
insights job ran 8 times a day and each run started an ~850MB chrome tree inside
the tomcat JVM's host -- a box holding -Xmx8g tomcat plus a -Xmx2g cron jar on
16GB that has been kernel-OOM-killed twice with tomcat the victim.
It was also losing data the whole time. Verified on prod across 7 consecutive
days: every row of cs.agent_daily_insight has logged_in_seconds, break_seconds,
available_seconds, talk_seconds, calls_answered, missed_calls and total_calls
set to 0. Two separate causes, both fixed here:
- parseTimeToSeconds split on ':' expecting 'HH:mm:ss', but the table renders
'2h 50m 39s'. A one-element split fell through to return 0, and because it
never reached the NumberFormatException branch it did not even warn. It now
parses the h/m/s shape and still accepts HH:mm:ss and HH:mm.
- the scrape never captured the call counts at all. The API carries them.
The auth chain is four calls and is not guessable, so it is documented in
KnowlarityApiClient: POST /vr/sr_login/ establishes the session and returns an
HS256 token that the API REJECTS; GET /newsr/user_details yields new_sr_ui_url
carrying a one-shot SSO blob (in a browser this hop is javascript, so it is
invisible to anything that merely follows redirects); POST /vr/sso_login/
exchanges that blob for the RS256 token the API accepts, whose claims embed the
srsessionid and so bind it to the session; GET /newsr/agents_insights/ with
header jwtAuthorization. Wrong token and wrong header name both answer
'Invalid token', so the error never tells you which mistake you made. The window
parameters are start_time/end_time -- start_date/end_date authenticates fine and
returns 'Error in API'.
Redirects are followed BY HAND. setInstanceFollowRedirects(true) exposes only
the final response's headers, so the Set-Cookie issued on the intermediate hops
is lost, the session never forms and user_details answers with an HTML error
page. This cost a debugging cycle; the reason is commented at the call site.
Verified against the live account before committing: 13 agents returned,
calls_offered == calls_answered + missed_calls holds for all 13, durations match
the previous scrape to within the elapsed window (~55s), and formatSeconds /
parseTimeToSeconds round-trip cleanly over the real values.
DTO fields stay the same display strings ('2h 50m 39s'), so every existing
reader is unaffected; only the previously-zero numeric columns change.
KnowlarityScraperService still uses ChromeDriver for operator-triggered break-log
scrapes, so the selenium dependency stays for now. Its @Scheduled annotations are
already commented out, so nothing unattended starts a browser any more. |
|
| 37677 |
30 m |
ranu |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/b2b/ |
price drop and hike fixed |
|
| 37676 |
37 m |
ranu |
/trunk/profitmandi-web/src/main/java/com/spice/profitmandi/web/controller/ |
v2 version some fixes |
|
| 37675 |
1 h 5 m |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Remove the dead Samsung and Amazon Selenium paths
Both were manual one-shots that nothing runs any more, and each started an
850MB chrome tree that now has to queue on the browser lane, so they are cost
without a caller.
Evidence they are dead rather than merely idle:
- No crontab entry, no cron.d file and no script references --samsung or
--amazonPurchase, and neither flag appears even once in cron.log.
- fofo.activated_imei has ZERO Samsung rows written by the cron (auth_id 0) in
the last 90 days. All 2,331 Samsung rows in that window are auth_id 307, i.e.
the human CSV upload, most recently 16-Sep. The scraper is not what keeps
Samsung current; people are.
- ScheduledSkeleton.fetchImeiActivation() had already been retired in place --
its @Scheduled was commented out with 'No longer scheduled'.
- RunOnceTasks.amazonPurchase() reads /Users/amit/Downloads/amazon.xlsx, a
laptop path that cannot exist on the server.
Removed: SamsungIMEIActivationService, the whole scheduled/amazon package
(AmazonPurchaseService, OrderSummary, OrderRow, AmazonUser), their RunOnceTasks
callers and helpers (fetchImeiActivation, amazonPurchase, getOrderSummary,
parseRow), the two Application CLI blocks and the retired ScheduledSkeleton
wrapper. The amazon package had no importers outside RunOnceTasks.
Both also leaked a chrome profile dir on every run -- AmazonPurchaseService
never called quit() at all, and SamsungIMEIActivationService called it outside
any finally -- so this removes two leak sources rather than fixing them.
Untouched: RunOnceTasks.mailDashboardScreenshots() is a third dead Selenium
one-shot (also zero invocations) but it mails a report, so it is left for a
separate decision. |
|
| 37674 |
2 h 44 m |
amit |
/trunk/profitmandi-fofo/src/main/java/com/spice/profitmandi/web/service/ |
Hold the browser lane around the knowlarity scrapes
These run Chrome inside the tomcat JVM, on the same box as the cron jar's
oppo/realme imei lane, with nothing coordinating the two. isScraping only guards
this JVM and cannot see the cron's driver, so the interlock has to be the
OS-level one -- BrowserLane, common r37672.
Fifteen minutes rather than the cron side's five: the insights scrape runs only
8 times a day against a lane that is busy ~96% of the time, so a short timeout
here would mean the insights never refreshed. On contention it serves cache.
The break-log scraper takes the lane around its freshDriver, which has a clean
try/finally bracket. setupDriver()'s long-lived field driver is deliberately left
alone -- its lifetime spans init to @PreDestroy and holding an OS lock that long
would starve the imei lane. |
|
| 37673 |
2 h 44 m |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Hold the browser lane around the oppo/realme/motorola drivers
Takes BrowserLane (common r37672) before new ChromeDriver and releases it after
quit() returns -- the 850MB is resident for the whole chunk, not just at
startup, so bracketing only the constructor would protect nothing.
Five minutes of waiting, then give up and return what we have: the only things
that can hold the lane that long are the other brand mid-chunk or the knowlarity
scrape in the fofo tomcat, and giving up costs nothing because an unstamped imei
stays pending and the lane's next turn picks it up.
CheckMotorolaWarrantyTask carries a comment saying 'do not let this job overlap
the oppo/realme window' that nothing ever enforced. It is wired here too so it is
already safe whenever it gets a trigger -- it still has none today.
Note on sizing, since the obvious knob is the wrong one: shrinking CHUNK from 25
was evaluated and rejected. The idle window is a fixed 20s bolted onto a variable
work period, so 25->10 moves the duty cycle only 96% -> 91% while costing 8.4%
of daily throughput and 2.5x the driver launches. The lever for duty cycle is the
fixedDelay gap, not the chunk size. |
|
| 37672 |
2 h 44 m |
amit |
/trunk/profitmandi-common/src/main/java/com/spice/profitmandi/common/util/ |
Add BrowserLane: machine-wide interlock so only one headless Chrome runs at a time
Every ChromeDriver tree on the app box costs ~850MB RSS, on a host holding a
-Xmx8g tomcat and a -Xmx2g cron jar on 16GB that has been kernel-OOM-killed
twice with tomcat the victim. Peak concurrent drivers is the number that
matters, not drivers per day.
Measured on the oppo/realme lane: a 25-imei chunk takes 505s (12s driver
startup, then ~19.7s an imei) against a fixed 20s fixedDelay, i.e. a 96% duty
cycle. Meanwhile tomcat starts its own driver 8 times a day for the knowlarity
scrape. On 17-Sep that scrape ran 11:40:00-11:41:07 and the lane's next driver
came up at 11:42:21 -- it cleared by 74 seconds, by luck. At a 96% duty cycle
the collision cannot be dodged by choosing better times, only by serialising.
The two JVMs are separate processes owned by different users, so an in-process
flag cannot see across the boundary. Uses an OS file lock at
/var/lock/sd-browser.lock (overridable via -Dsd.browser.lock), created 0666 on
first use because cron runs as root and tomcat as tomcat and whichever starts
first creates it. A file lock rather than a marker file so the kernel reclaims
it when the holder dies: verified by SIGKILLing the holder mid-hold, the next
JVM acquired in 264ms. A ReentrantLock gates it first, since a second thread in
the same JVM would otherwise get OverlappingFileLockException rather than block.
Callers wait rather than skip. The knowlarity scrape needs 67s against a lane
busy 96% of the time, so skip-on-contention would mean it essentially never
refreshed again. Eight scrapes a day costs the imei lane about nine minutes of
waiting in total.
Fails open if the lock file is unusable (missing /var/lock, read-only mount, dev
laptop): serialising within the JVM only, rather than stopping imei activation
over what is a safety interlock. |
|
| 37671 |
4 h 23 m |
ranu |
/trunk/ |
loi revival process modify |
|
| 37670 |
19 h 27 m |
amit |
/trunk/profitmandi-fofo/src/main/ |
Check GSTIN shape on the onboarding screen before submit (jsVersion 425)
Both GST boxes are validated client-side and must match, so ops see the problem on the screen
instead of discovering it when NIC refuses the first invoice. Server-side guard in dao r37669
is what actually enforces it. |
|
| 37669 |
19 h 27 m |
amit |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/service/user/ |
Reject an invalid or mismatched partner GSTIN before it is saved
The onboarding screen carries the GSTIN in two boxes both labelled GST NUMBER - the retailer
one (dtr.retailer.number) and the store one (fofo_store.gst_number, which billing reads).
Neither was validated, so DLSH1660 was saved with the LOI's pincode in the store box while the
retailer box held the real GSTIN, and its first invoice would have been refused by NIC.
validateGstNumbers runs before any write (covers the fofo portal, the V2 API and the partner
app, which all funnel through updateRetailerDetails): both values are normalised, both must be
valid GSTINs, and when both are present they must agree - across 1,800 partners on record they
already do, bar the one that caused this. Needs common r37668. |
|
| 37668 |
19 h 27 m |
amit |
/trunk/profitmandi-common/src/main/ |
Validate GSTIN shape, state code and checksum
isValidGstNumber accepted any 15-character string, so a pincode (110095, rejected only for
length) and junk (Hdjiekwbdbsjskz) reached fofo_store.gst_number, which billing sends to NIC
as the recipient - the invoice is then refused and the goods cannot ship.
Now: 2-digit state code + PAN + entity + Z + mod-36 check digit, trimmed and upper-cased.
Blank still passes (the field is optional); callers needing one present check that themselves.
Checked against live data: 1,569 of 1,570 well-formed partner GSTINs pass, the one that fails
(HRSP330) has a genuinely wrong check digit.
Adds RTLR_1020 / RTLR_1021 for the invalid and mismatched cases. |
|
| 37667 |
20 h 35 m |
amit |
/trunk/profitmandi-dao/src/main/resources/sql/ |
chore(sql): drop vendor catalog pricing for supplier 1 Hotspot, flagged internal 2026-09-16 (as run on prod; backups _bak_vcp/_vcpl_internal_hotspot_20260916) |
|
| 37666 |
21 h 0 m |
amit |
/trunk/profitmandi-web/src/main/java/com/spice/profitmandi/web/controller/ |
Hot Deal brand: /fofo/hotDeals/brands accepts categoryId, so the brand chips match the active tab |
|
| 37665 |
21 h 0 m |
amit |
/trunk/profitmandi-common/src/main/java/com/spice/profitmandi/common/solr/ |
Hot Deal brand: optional categoryId on the brand facet, so the chips can be scoped to one tab |
|
| 37664 |
21 h 8 m |
ranu |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/dao/repository/user/ |
loi new revival process |
|
| 37663 |
21 h 37 m |
vikas |
/trunk/profitmandi-fofo/src/main/java/com/spice/profitmandi/web/controller/ |
Added exception to method sign |
|
| 37662 |
21 h 44 m |
vikas |
/trunk/profitmandi-dao/src/main/java/com/spice/profitmandi/service/lms/ |
LMS click to call |
|
| 37661 |
21 h 46 m |
amit |
/trunk/profitmandi-common/src/main/java/com/spice/profitmandi/common/web/client/ |
Fix trunk: RestClient was left behind by r37655
r37655 deleted MandiiResponse but omitted RestClient from the commit, so
trunk did not compile - RestClient still imported the deleted class.
RestClient now uses RawHttpResponse: executeMandii is renamed executeRaw and
postWithResponse returns the new type. The dead getMandii helper is removed;
its only caller was MandiiService. |
|
| 37660 |
22 h 6 m |
ranu |
/trunk/ |
ticket download option given and some enhancement on notification panel |
|
| 37659 |
22 h 35 m |
amit |
/trunk/profitmandi-fofo/src/main/java/com/spice/profitmandi/web/ |
Remove the Mandii callback from fofo (r37655)
Drops the /mandii payment-notification stub from HdfcPaymentController - it
only logged the body - and its three entries in the WebConfig auth whitelist.
PartnerOnBoardingPanelController imports follow services.mandii ->
services.kyc for Gender/MaritalStatus/AddressProofType. |
|