| Rev |
Age |
Author |
Path |
Log message |
Diff |
| 37417 |
48 d 14 h |
vikas |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Whatsapp botpenguin |
|
| 37416 |
48 d 14 h |
vikas |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Port two cron WhatsApp call sites left behind by r37409/r37415.
OrderTrackingService sends the ORDER_DELIVERED template with the airway bill
as its only variable; the push notification on the same loop keeps its
formatted sentence, since template wording lives in Meta, not the call site.
Application drops the userMobileNumberOptIn and authUserMobileNumberOptIn CLI
options, whose NotificationService.optIn backing went with the Gupshup gateway.
ScheduledTasks.java is deliberately not in this commit: locally it also carries
unrelated in-flight remark-scoring work whose package is still unversioned, so
committing it would put @Autowired RemarkScoreRunner into trunk with no such
class. Its two WhatsApp call sites (optIn at ~4618, sendWhatsappMessage at
~5276) therefore still fail to compile at HEAD and need a follow-up. |
|
| 37410 |
48 d 14 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Oppo warranty check: log to the logger instead of stdout
Oppo has resolved nothing all day -- 83 ticks, the same 10 IMEIs recycled every
time, zero "Serial Number ... Date" results -- and the log shows only a single
line, 'Initiating webdriver...'. That is not silence: the class had 1 LOGGER call
against 28 System.out.println, and the cron JVM's stdout is a socket inherited
from the SSH session that launched it. Every diagnostic was going nowhere.
So the code has been reporting what is wrong 28 times per attempt (which element
was missing, the computed slider distance, success or failure per attempt, the
parsed date) and all of it was discarded. Same pattern as the Vivo bug found
today, where a five-word log line hid the real cause for weeks.
Converted to LOGGER at info, with warn on the failure paths and error on the
catch-all, which now includes the exception rather than a bare printStackTrace.
The slider-widget timeout says which element it was waiting for, since that is
the most likely failure point.
No behaviour change -- only where the output goes. |
|
| 37408 |
48 d 15 h |
amit |
/trunk/ |
Vivo IMEI activation: stop spending captchas on line items with no IMEI
76% of production captcha rejections were line items whose serial number is
null. Vivo answers those with {"msg":"参数为空"} -- "parameter is empty" --
and status 0, which this code recorded as a captcha failure. So a correctly
solved captcha looked wrong, no activated_imei row was written, the line item
stayed pending, and it came back every 5 minutes indefinitely. Those rows were
permanently consuming roughly 43% of the run quota, which is why every tick ran
full at 20/20 and the backlog never drained.
It also made the model look far worse than it is: measured accept rate 44%,
while the same solver scores 85-93% when the IMEI is present. True captcha
accuracy is around 77%.
Fixed at source: both named queries now exclude null and blank serial numbers,
so such line items never enter the pool (this also covers the Realme caller).
The loop additionally skips them before fetching a captcha, so no captcha,
solver call or Vivo request is spent discovering it.
Found via the diagnostics added in r37407 -- the previous log line recorded
five words and discarded the response that named the cause. |
|
| 37407 |
48 d 15 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Vivo IMEI activation: log why a captcha was rejected
Production refuses ~56% of submissions, but reading the stored images by hand
showed 9 of 10 carried the CORRECT code. Replicating the request from the same
box gives 85-93%, and every external difference was measured and ruled out:
real vs dummy imei (7/8 both), user-agent (83% both), session reuse (74 vs 75%),
two concurrent jobs (80% both), source IP, and fetch-to-submit delay from 0 to
10s (90-91% throughout). Vivo's reply is byte-identical in every failure, so the
payload carries no discriminator.
That leaves something inside this process, and the old log line recorded five
words and discarded the evidence. It now captures the code submitted, the imei,
the real fetch-to-submit latency, the session cookies actually held at submit
time, and Vivo's full response.
The cookies matter most: if the SESSION cookie is being dropped or rotated
between fetching the captcha and submitting it, the captcha would be validated
against the wrong session and refused despite a correct code -- which fits every
measurement above and is invisible from outside the JVM. |
|
| 37406 |
48 d 16 h |
amit |
/trunk/profitmandi-cron/src/main/resources/META-INF/ |
Vivo captcha: put the /verdict shared secret in the cron properties
The token was only in run-cron.sh, which is not version-controlled. Rebuild the
box or deploy elsewhere and it silently vanishes -- the verdict call is
best-effort and swallows failures, so nothing would break loudly; training data
would just quietly stop being collected. Properties travel with the build.
dev and staging are deliberately empty: with no token CaptchaService skips the
call entirely, so a local run cannot post verdicts to the live solver and
pollute the training corpus.
CAPTCHA_VERDICT_TOKEN in the environment still overrides this, so the token can
be rotated without a rebuild. |
|
| 37402 |
49 d 7 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Vivo IMEI activation: report Vivo's captcha verdict back to the solver
Only this cron ever learns whether Vivo accepted a captcha (status 0 means it
was wrong), so it is the only place that can label the solver's training data.
After each checkCode response it now posts the image, the code and the outcome
to the solver's /verdict endpoint, which files the sample as accepted (the
prediction was right - a free label) or rejected (needs a human to label it).
The solver has been running at a measured 22.3% accept rate and no training
data was ever collected, so the model could not be improved at all.
Strictly best-effort: 2s connect / 3s read timeouts and every exception
swallowed. Data collection must never slow down or break IMEI activation.
The shared secret is read from CAPTCHA_VERDICT_TOKEN in the environment,
falling back to a captcha.verdict.token property, so it need not be committed.
Without a token the call is skipped entirely.
Also drops /tmp/captcha.jpg: the captcha bytes are needed in memory for the
verdict report anyway, so the two concurrently-scheduled Vivo jobs no longer
share one file and overwrite each other's image. |
|
| 37401 |
49 d 7 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Drop commented reference to the removed InvoiceFormatter |
|
| 37395 |
49 d 7 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
observability: retire the dead Nagios monitoring path, migrate balance signals to Micrometer
There is no Nagios server and the NRPE daemons have been removed from every
host, so the properties files written under /var/log/services were being
produced for nobody.
Deleted:
- CronMetricsService: zero references. Spring instantiated the @Service so its
constructor registered three meters, but nothing ever incremented them --
cron_execution_* read 0 in prometheus while cron_job_count_total (from
CronJobMonitorAspect) had recorded 1,597 executions.
- NagiosMonitoringScheduledSkeleton.
Removed the nagios properties writes from ScheduledSkeleton (-155 lines) and
KnowlarityCallMonitorScheduler (-46, including its now-purposeless
markTaskComplete helper). These recorded a per-job 0/1 status that
CronJobMonitorAspect already captures properly as success, failure and
duration metrics.
Kept the two signals that are genuinely useful: SMS gateway balance and the
ThinkWalnut recharge wallet balance now publish as Micrometer gauges via the
new BalanceGauges component, scraped from /actuator/prometheus. -1 means 'not
read yet' so a scrape before the first run is distinguishable from a real zero.
No behaviour change to any scheduled job. Compiles clean. |
|
| 37393 |
49 d 11 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Vivo IMEI activation: validate captcha solver response before submitting to Vivo
The captcha solver at 45.79.121.178 was down from 09-Jul-2026 to 25-Aug-2026
(uwsgi never restarted after a host reboot). nginx returned a 502 page for
every request. RestClient.executeJson returns the response body regardless of
HTTP status, so CaptchaService handed that HTML page to Vivo as the captcha
code and Vivo logged it as 'Found invalid captcha' - indistinguishable from an
ordinary wrong guess. The outage went unnoticed for 46 days.
CaptchaService now checks the response against the solver model's own 31-class
alphabet (^[1-9A-HK-NP-Z]{4}$) and returns null for anything else, logging the
offending payload. VivoImeiActivationService skips the IMEI when the code is
null instead of posting the garbage; it is retried on the next run.
Also drops a System.out.println of the full base64 captcha image. |
|
| 37369 |
54 d 16 h |
amit |
/trunk/ |
Fixed mail sender everywhere |
|
| 37368 |
55 d 7 h |
amit |
/trunk/profitmandi-cron/src/main/ |
Cron: raise SMTP timeouts, rotate app password, add offer-circular ingest keys
- SMTP 10s -> 30s connect / 120s read-write on both senders; the 10s read timeout was
cutting off larger attachment sends. App password rotated to match the current
Google account credential.
- offer.circular.ingest.enabled / offer.circular.review.url added to dev, staging and
prod. Off by default in all three: enabling it makes the job pick up DRAFT circulars
uploaded from the FOFO portal, parse them and mail the uploader, which should only be
switched on once offer.circular.dir is a real path in that environment. There is no
fallback between profiles, so the key has to exist in each one. |
|
| 37352 |
55 d 15 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Fixed max limit to 15 lac for Credit limit |
|
| 37351 |
55 d 16 h |
ranu |
/trunk/ |
code committed for sales l3 added in bi and other model |
|
| 37340 |
56 d 13 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Fixed max limit to 15 lac for Credit limit |
|
| 37339 |
56 d 14 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Fixed max limit to 15 lac for Credit limit |
|
| 37338 |
56 d 14 h |
amit |
/trunk/profitmandi-cron/ |
Remove offer-circular ingest from cron - it now runs in the portal
Counterpart to the fofo commit that moved the parse into profitmandi-fofo. Nothing is
lost: all 13 parser classes and the test moved verbatim, and CircularIngestScheduler
was replaced by an executor-driven runner in the portal.
- com.smartdukaan.cron.offercircular deleted, main and test
- tabula dependency removed; PDFBox no longer enters this artifact at all
- dumpCircularClasspath moved to fofo, where the parser and its classpath now live
This module no longer knows anything about offer circulars, so a cron rebuild is no
longer a prerequisite for the feature to work - which was the entire problem. |
|
| 37334 |
56 d 15 h |
amit |
/trunk/profitmandi-cron/ |
Sweep stalled offer circulars before each ingest run, and drop stale scaffolding
- The scheduler now calls reclaimStalledProcessing before looking for new work, so
a circular stranded in PROCESSING by a dead or redeployed worker is failed with a
reason instead of staying invisible forever. Threshold is 30 minutes against a
parse that takes seconds, so it can only ever catch a genuinely dead worker. The
sweep is wrapped so it can never stop the actual ingest.
- dumpCircularClasspath was labelled a temporary helper for diffing against the
reference Python. That Python has been deleted, but the task is what lets an
ingest be reproduced and measured locally per scripts/offer_circular/README.md,
so the comment now says what it is for rather than telling the next reader to
delete it.
- CircularExtractor's javadoc claimed verification against the Python reference.
That proved equivalence, not correctness - both shared the missing-memory-unit
bug that bound offers to the wrong SKU. Reworded so it cannot be read as a
correctness guarantee, and points at the fixture and ProductNamesTest instead.
Verified: full ingest of the Aug'26 circular is byte-identical to the reference -
239 offers, 663 products, 279 AUTO_EXACT. ProductNamesTest 11/11. |
|
| 37330 |
57 d 6 h |
amit |
/trunk/profitmandi-cron/ |
Add Pine Labs affordability circular parsing and ingest
Turns the monthly OEM "Mobile & Laptop Offers" PDF into the offers schema: tabula
extraction to nine verbatim columns per row, per-cell parsers for benefit, tenure,
bank and footnote text, catalog matching, and a scheduler that claims DRAFT
documents uploaded from the FOFO portal and emails the uploader the outcome.
Ships inert. offer.circular.ingest.enabled defaults to false, so the scheduler does
nothing until an environment opts in, and offer.circular.review.url defaults to
empty - neither key is required for the context to start.
- tabula added here and not in profitmandi-common so PDFBox never reaches the
web/fofo WARs; bouncycastle, slf4j-simple and jai-imageio excluded (version
clash, duplicate SLF4J binding, and unused image decoding respectively)
- offer_raw_row holds all nine columns verbatim, so re-parsing reads the table and
never the PDF again
- product_alias is consulted before matching, so a human confirmation recorded once
keeps applying every following month
ProductNamesTest locks in the product-name parsing, which decides which SKU an
offer's money lands on. Every case there is a real mis-parse, and they share one
root cause: characters or digits belonging to the model name being eaten as memory
or stripped as punctuation. Two worth naming:
- A memory spec with no GB/TB unit is still a memory spec. Motorola writes
"(8+256)" and vivo "(8+256G)"; unrecognised, the matcher believed no size was
given and bound the offer to an arbitrary sibling - a 1,000 Edge 60 Pro 8+256
offer and a 2,000 12+256 offer landed on the same SKU.
- Two variant groups written back-to-back are two products. "Edge 70 Pro
(8+256)(12+256)" stayed one product bound to a single SKU while the 12+256
variant silently got no offer at all.
Bundled accessories are deliberately NOT stripped back to the bare phone. Doing so
resolves 30 CATALOG_GAP rows and looks safe on Oppo, whose bundled and bare rows
carry identical values - but vivo caps X300 Pro(16+512G) at 10,000 on its own row
and 11,000 on the "+Extender" row, the difference being the Extender. Merging them
would let the bundle's cap be claimed on a phone sold without the accessory.
Whether a bundle offer transfers to the bare SKU is a commercial question the PDF
does not answer, so it is a manual coverage decision, not a parsing rule.
Verified against the Aug'26 circular on the local DB: 324 rows in the PDF, 239
loaded, producing 361 benefits, 511 tenures, 911 bank links and 663 products, of
which 74% resolve automatically. ProductNamesTest 11/11. |
|
| 37326 |
57 d 10 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Add --backfillMissingEwb cron arg
Runs InvoiceService.backfillMissingEwb over an explicit invoice list:
--backfillMissingEwb --invoiceNumbers=A,B,C [--dryRun]
Explicit list rather than a date window so the backfill only touches the invoices known
to have lost their EWB. NOT_SUPPORTED on the RunOnceTasks wrapper suspends the
class-level transaction, keeping the NIC lookups outside one. |
|