| Rev |
Age |
Author |
Path |
Log message |
Diff |
| 37429 |
48 d 0 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Oppo: wait for the lookup response instead of reading it once
Oppo read window.__oppoResp immediately after the success signal, which races the
request that produces it: in the 20:00 hour 17 successes yielded 16 'no
getDeviceInfo response' and a single lookup, against 673 lookups and 0 misses
earlier in the day.
Now polls for up to 8s, matching Realme (r37426). A failed read also reports the
captcha widget's own verdict, so a miss says whether the drag was refused
(POSITION_MISMATCH) or passed and the lookup simply did not arrive.
Widget verdicts confirm the calibration work: 14 success:true against 18
POSITION_MISMATCH is 43% real aim accuracy, up from ~9% before r37422. |
|
| 37428 |
48 d 0 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Oppo/Realme: record the captcha widget's own verdict alongside the lookup
r37426 made the lookup response the success signal, which is authoritative but
silent about WHY a drag was refused -- a timeout looks identical whether the aim
was wrong, the widget never rendered, or the page changed again.
The captcha widget answers that itself: it calls captcha-ind-sec.heytapmobile.com
and its replies carry a status and message. The hook now keeps the last such
response, and a failed attempt logs it, so 'Failed = 3' becomes 'Failed = 3
(captcha said: ...)'.
Note what the widget does NOT give us: its init call returns the puzzle's y
coordinate but not the x, which is the secret being protected. So the circle
detection still has to find the horizontal target -- there is no shortcut there,
and that remains the largest loss (60% of Oppo attempts never find the circles). |
|
| 37426 |
48 d 0 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Realme: treat the lookup response, not a missing error element, as success
After r37425 Realme logged 27 'Success at attempt' and 27 'no active/check
response captured' in the same window, and wrote 0 rows. The hook was not at
fault -- reproduced locally, it captures both XHR and fetch correctly.
Success was being inferred from the ABSENCE of dx_captcha_basic_bar-inform:
try { wait5Sec.until(visibilityOf(bar-inform)); log('Failed') }
catch { log('Success') }
realme rebuilt this page, so that element may no longer exist. The wait then
always times out and EVERY attempt reports success while the captcha was never
solved, no lookup fired and nothing was captured. The 27/27/0 is that exactly.
The response arriving is the only signal that actually proves the captcha
passed, so it now waits for that (polling, up to 8s) and treats its absence as a
failure worth retrying. This also removes the separate 'no response captured'
branch, since reaching the success path now means the response is in hand.
Oppo is left alone: its bar-inform detection is working (0 missed captures
against 673 lookups), so there is nothing to fix there. |
|
| 37425 |
48 d 0 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Oppo/Realme: capture the response from fetch as well as XHR
r37423 got Realme solving the captcha again -- 4 of 4 on the first tick after
deploy -- but every one logged 'Result shown but no active/check response
captured'. The capture hook only wrapped XMLHttpRequest, and realme's page issues
the lookup through fetch, so it recorded nothing and each imei was stored with a
null date.
The hook now wraps window.fetch too, cloning the response before reading it so
the page still consumes its own body. Applied to Oppo as well: its XHR path is
working today, but the same silent failure would follow any move to fetch, and
the cost is a few lines. |
|
| 37424 |
48 d 1 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Oppo/Realme slider captcha: size the calibration nudge from the gap
r37422 replaced a 15px calibration nudge with a fixed 60px, which fixed the
under-measurement but introduced the opposite error. Production data over 764
calibrations shows 45% ended UNUSABLE (closed <= 0, i.e. the piece shot past the
hole and the measured gap grew instead of shrinking), and among gaps under 60px
it was 85%.
The nudge is now sized from the gap -- aim to close about half of it -- and
clamped to [10,60]. Replaying the real observed gap distribution: unusable and
overshoot fall from 21% to 2%.
The observed travel ratio in production is 1.13 circle px per slider px, against
1.68 measured locally; it varies with render scale, which is why the ratio is
still derived from the measurement. ASSUMED_RATIO only sizes the probe, and 1.7
errs toward a smaller nudge, which is the safe direction here.
Oppo is currently solving 43% (was 9% before r37422). Realme carries the same
widget and the same defect and has not been deployed yet. |
|
| 37423 |
48 d 1 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Realme warranty check: retarget the rebuilt page and read its JSON
Realme has recorded nothing since 24-Aug, the same day Oppo stopped. Driving the
page confirms why: it was rebuilt and every selector the code used is gone.
.sn-input -> the field is now .imei-input input (placeholder 'Please input IMEI')
div.check-btn -> the submit is now an icon, .imei-input .search-icon
So it failed even earlier than Oppo did -- it could not type the imei, let alone
reach the captcha, which is why the log showed only 'Initiating webdriver...'.
The result now comes from the JSON the page already receives
(customer-api.realme.com/in/active/check) rather than an XPath keyed off a label
that no longer exists. That old locator also had an '| //h1[contains(@class,
title)]' branch which would silently match a page heading and try to parse it as
a date. The API is explicit: expiryDate (yyyy.MM.dd) and an isActivation flag,
so nothing is inferred from message text.
expiryDate is the WARRANTY EXPIRY, so a year is still subtracted -- warranty is
12 months as standard. Checked against our own records rather than assumed:
41,267 of 41,289 stored Realme activation dates precede the date we checked
them, averaging 13 days before, which is only consistent with the subtraction
already being correct.
Ports the r37422 captcha fix verbatim -- identical dingxiang slider, identical
15px calibration nudge that made it land ~30px out against a ~30px hole. And as
with Oppo the outcome is always recorded, even as null: findElement THREW when
the element was missing, so nothing was written and the imei was re-queued every
5 minutes indefinitely.
Also converts 29 System.out.println to the logger. The cron JVM's stdout is a
socket from the launching ssh session, so every diagnostic was going nowhere. |
|
| 37422 |
48 d 3 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Oppo slider captcha: calibrate over a longer nudge and correct before releasing
The captcha solves about 10 times in 115 attempts. The arithmetic was right; the
measurement feeding it was not.
To place the piece the code must first learn how far it travels per slider pixel,
which it did by nudging 15px and re-measuring. The piece moves ~1.7px per slider
px, so 15px closed the gap by only ~25px -- and Hough circle detection carries
about +/-2-3px per circle, so the ratio came from a measurement roughly 20%
wrong. That error was then multiplied across the whole remaining travel: on a
150px gap it landed ~30px out, against a hole ~30px wide.
Nudging 60px closes ~100px, so the same detection noise is ~5% rather than ~20%.
The code also released on its single computed offset, committing all of that
error to the final position; it now measures once more after the main move and
closes the remainder before releasing. Math.round replaces an (int) cast that
biased every move short by up to a pixel.
Monte Carlo over the measurement noise (20k trials, +/-8px tolerance): median
placement error 16.3px -> 2.1px, success 26% -> 98%. Holds across a 2x range of
travel ratios and degrades gracefully as detection worsens, because the ratio is
derived rather than assumed. Real-world will be lower -- the same model puts the
old code at 26% where production sees 9% -- so expect roughly 1 in 3, not 49 in
50, and read Success/Failed in the log to get the true figure.
The ratio is returned rather than held on a field: this is a singleton and the
Oppo secondary and tertiary jobs run through it concurrently. |
|
| 37420 |
48 d 4 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Oppo warranty check: read the JSON response instead of scraping the date
Oppo has recorded no activation dates since 23-Aug. With logging restored
(r37410) the cause is visible: the captcha IS solved -- 10 successes per tick --
and it then fails on the very next line with
no such element: //*[contains(text(),'UTC+5.5') or contains(text(),'Non-Activate')]
Their result is now label/value pairs, and the value is rendered from TWO text
nodes: dayjs(regDate).format('DD/MM/YYYY') plus a '(UTC+x)' suffix computed from
the browser's own timezone. XPath contains(text(),...) tests only the first node
-- the date -- so it never matched. 'Non-Activate' is gone too: an inactive
device now reads 'The system will update the date within 7 days of device
activation.' Verified against a replica of their DOM: old locator 0 matches, and
the value element yields '23/08/2026 (UTC+5.5)'.
Rather than chase their markup again, take the value from the source: the page
calls /oppo-api/basic/v1/getDeviceInfo and the response carries regDate as epoch
millis -- no date format, no locale, no timezone suffix to parse. A small hook
records that response as the page receives it; nothing extra is requested and no
captcha behaviour changes.
Also switches the outcome to always record the imei. findElement THREW when the
element was missing, which skipped the fallback entirely, so nothing was written
to dateMap and the row was re-queued on every run indefinitely -- 'Could not
capture date' appears 0 times in the log because it was unreachable. |
|
| 37419 |
48 d 6 h |
vikas |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/remarkscore/ |
Whatsapp botpenguin |
|
| 37418 |
48 d 6 h |
vikas |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/remarkscore/ |
Whatsapp botpenguin |
|
| 37417 |
48 d 6 h |
vikas |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Whatsapp botpenguin |
|
| 37416 |
48 d 6 h |
vikas |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Port two cron WhatsApp call sites left behind by r37409/r37415.
OrderTrackingService sends the ORDER_DELIVERED template with the airway bill
as its only variable; the push notification on the same loop keeps its
formatted sentence, since template wording lives in Meta, not the call site.
Application drops the userMobileNumberOptIn and authUserMobileNumberOptIn CLI
options, whose NotificationService.optIn backing went with the Gupshup gateway.
ScheduledTasks.java is deliberately not in this commit: locally it also carries
unrelated in-flight remark-scoring work whose package is still unversioned, so
committing it would put @Autowired RemarkScoreRunner into trunk with no such
class. Its two WhatsApp call sites (optIn at ~4618, sendWhatsappMessage at
~5276) therefore still fail to compile at HEAD and need a follow-up. |
|
| 37410 |
48 d 7 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Oppo warranty check: log to the logger instead of stdout
Oppo has resolved nothing all day -- 83 ticks, the same 10 IMEIs recycled every
time, zero "Serial Number ... Date" results -- and the log shows only a single
line, 'Initiating webdriver...'. That is not silence: the class had 1 LOGGER call
against 28 System.out.println, and the cron JVM's stdout is a socket inherited
from the SSH session that launched it. Every diagnostic was going nowhere.
So the code has been reporting what is wrong 28 times per attempt (which element
was missing, the computed slider distance, success or failure per attempt, the
parsed date) and all of it was discarded. Same pattern as the Vivo bug found
today, where a five-word log line hid the real cause for weeks.
Converted to LOGGER at info, with warn on the failure paths and error on the
catch-all, which now includes the exception rather than a bare printStackTrace.
The slider-widget timeout says which element it was waiting for, since that is
the most likely failure point.
No behaviour change -- only where the output goes. |
|
| 37408 |
48 d 7 h |
amit |
/trunk/ |
Vivo IMEI activation: stop spending captchas on line items with no IMEI
76% of production captcha rejections were line items whose serial number is
null. Vivo answers those with {"msg":"参数为空"} -- "parameter is empty" --
and status 0, which this code recorded as a captcha failure. So a correctly
solved captcha looked wrong, no activated_imei row was written, the line item
stayed pending, and it came back every 5 minutes indefinitely. Those rows were
permanently consuming roughly 43% of the run quota, which is why every tick ran
full at 20/20 and the backlog never drained.
It also made the model look far worse than it is: measured accept rate 44%,
while the same solver scores 85-93% when the IMEI is present. True captcha
accuracy is around 77%.
Fixed at source: both named queries now exclude null and blank serial numbers,
so such line items never enter the pool (this also covers the Realme caller).
The loop additionally skips them before fetching a captcha, so no captcha,
solver call or Vivo request is spent discovering it.
Found via the diagnostics added in r37407 -- the previous log line recorded
five words and discarded the response that named the cause. |
|
| 37407 |
48 d 7 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Vivo IMEI activation: log why a captcha was rejected
Production refuses ~56% of submissions, but reading the stored images by hand
showed 9 of 10 carried the CORRECT code. Replicating the request from the same
box gives 85-93%, and every external difference was measured and ruled out:
real vs dummy imei (7/8 both), user-agent (83% both), session reuse (74 vs 75%),
two concurrent jobs (80% both), source IP, and fetch-to-submit delay from 0 to
10s (90-91% throughout). Vivo's reply is byte-identical in every failure, so the
payload carries no discriminator.
That leaves something inside this process, and the old log line recorded five
words and discarded the evidence. It now captures the code submitted, the imei,
the real fetch-to-submit latency, the session cookies actually held at submit
time, and Vivo's full response.
The cookies matter most: if the SESSION cookie is being dropped or rotated
between fetching the captcha and submitting it, the captcha would be validated
against the wrong session and refused despite a correct code -- which fits every
measurement above and is invisible from outside the JVM. |
|
| 37406 |
48 d 9 h |
amit |
/trunk/profitmandi-cron/src/main/resources/META-INF/ |
Vivo captcha: put the /verdict shared secret in the cron properties
The token was only in run-cron.sh, which is not version-controlled. Rebuild the
box or deploy elsewhere and it silently vanishes -- the verdict call is
best-effort and swallows failures, so nothing would break loudly; training data
would just quietly stop being collected. Properties travel with the build.
dev and staging are deliberately empty: with no token CaptchaService skips the
call entirely, so a local run cannot post verdicts to the live solver and
pollute the training corpus.
CAPTCHA_VERDICT_TOKEN in the environment still overrides this, so the token can
be rotated without a rebuild. |
|
| 37402 |
48 d 23 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Vivo IMEI activation: report Vivo's captcha verdict back to the solver
Only this cron ever learns whether Vivo accepted a captcha (status 0 means it
was wrong), so it is the only place that can label the solver's training data.
After each checkCode response it now posts the image, the code and the outcome
to the solver's /verdict endpoint, which files the sample as accepted (the
prediction was right - a free label) or rejected (needs a human to label it).
The solver has been running at a measured 22.3% accept rate and no training
data was ever collected, so the model could not be improved at all.
Strictly best-effort: 2s connect / 3s read timeouts and every exception
swallowed. Data collection must never slow down or break IMEI activation.
The shared secret is read from CAPTCHA_VERDICT_TOKEN in the environment,
falling back to a captcha.verdict.token property, so it need not be committed.
Without a token the call is skipped entirely.
Also drops /tmp/captcha.jpg: the captcha bytes are needed in memory for the
verdict report anyway, so the two concurrently-scheduled Vivo jobs no longer
share one file and overwrite each other's image. |
|
| 37401 |
48 d 23 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Drop commented reference to the removed InvoiceFormatter |
|
| 37395 |
48 d 23 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
observability: retire the dead Nagios monitoring path, migrate balance signals to Micrometer
There is no Nagios server and the NRPE daemons have been removed from every
host, so the properties files written under /var/log/services were being
produced for nobody.
Deleted:
- CronMetricsService: zero references. Spring instantiated the @Service so its
constructor registered three meters, but nothing ever incremented them --
cron_execution_* read 0 in prometheus while cron_job_count_total (from
CronJobMonitorAspect) had recorded 1,597 executions.
- NagiosMonitoringScheduledSkeleton.
Removed the nagios properties writes from ScheduledSkeleton (-155 lines) and
KnowlarityCallMonitorScheduler (-46, including its now-purposeless
markTaskComplete helper). These recorded a per-job 0/1 status that
CronJobMonitorAspect already captures properly as success, failure and
duration metrics.
Kept the two signals that are genuinely useful: SMS gateway balance and the
ThinkWalnut recharge wallet balance now publish as Micrometer gauges via the
new BalanceGauges component, scraped from /actuator/prometheus. -1 means 'not
read yet' so a scrape before the first run is distinguishable from a real zero.
No behaviour change to any scheduled job. Compiles clean. |
|
| 37393 |
49 d 3 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Vivo IMEI activation: validate captcha solver response before submitting to Vivo
The captcha solver at 45.79.121.178 was down from 09-Jul-2026 to 25-Aug-2026
(uwsgi never restarted after a host reboot). nginx returned a 502 page for
every request. RestClient.executeJson returns the response body regardless of
HTTP status, so CaptchaService handed that HTML page to Vivo as the captcha
code and Vivo logged it as 'Found invalid captcha' - indistinguishable from an
ordinary wrong guess. The outage went unnoticed for 46 days.
CaptchaService now checks the response against the solver model's own 31-class
alphabet (^[1-9A-HK-NP-Z]{4}$) and returns null for anything else, logging the
offending payload. VivoImeiActivationService skips the IMEI when the code is
null instead of posting the garbage; it is retried on the next run.
Also drops a System.out.println of the full base64 captcha image. |
|