| Rev |
Age |
Author |
Path |
Log message |
Diff |
| 37485 |
21 h 50 m |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
carlcare: enable tecno alongside itel
Itel went first because it is the brand we can CHECK -- the DCR pull independently
produces itel dates to reconcile against, where tecno has no second source at all.
That check has passed:
- a 50-imei read-only trial answered 50/50, no errors, no sign rejections, and
agreed with our catalog brand on all 50
- 18 of the 50 carried a date, and every one of those 18 fell AFTER our own
billing date (3 to 1,111 days, median ~220)
- the first live tick after deploy returned the same rate: 16 dates, 32 not-yet-
activated, 1 malformed response absorbed by the has("status") guard
Tecno is also the reason this class exists. Itel was already served by the DCR pull;
tecno has been served by nothing since 2024-11-20 and has been manual CSV ever since.
Capacity: ~10,400 itel + ~1,890 tecno pending against 14,400 lookups a day. The
round-robin splits a tick four ways only while all four queues have work; tecno is
much the smaller pool, so it is exhausted a few hours in, its queues then come back
empty and itel gets the full 50 again for the rest of the day. Itel still clears
daily -- roughly 1,875 + 10,650 = 12,525 itel lookups against a 10,401 pool. |
|
| 37484 |
21 h 50 m |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/monitored/ |
cron metrics: register the status gauge as -1, not 0
The gauge was created when a job STARTS, and 0 means failure, so any job still
running its first execution after a restart reported FAILED. Selenium-driven and
report jobs take minutes, so this is not a narrow window -- it put ten jobs on
the dashboard as failed after the 2026-08-31 restart when none had failed.
-1 means 'has not finished a run yet'. CronJobFailing matches == 0, so those
jobs are simply absent from the alert until they genuinely complete once. |
|
| 37482 |
22 h 8 m |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
carlcare: recover tecno/itel activation dates, and fix trunk broken by r37479
r37479/r37480 swept a working-copy edit to Application.java into the mail commit: trunk
has called scheduledTasks.checkCarlcareImeiActivation() since then without containing it,
so profitmandi-cron has not compiled. This adds the rest.
WHAT THIS IS
The tecno feed has been dead since 2024-11-20, when transsion decommissioned the SAP
OData hosts cms.tecno-mobile.com:8099 / cms.itel-mobile.com:8099. The imwav DCR portal
recovers itel but has nothing for tecno -- all three tecno logins together expose 17
imeis over five years, against 31,928 tecno rows in fofo.activated_imei. Everything since
has been manual CSV via /imei/upload.
Carlcare is transsion's own after-sales arm, and the warranty-check page on carlcare.in is
backed by a public per-imei endpoint that answers for tecno, itel and infinix with no
login, no cookie and no captcha:
GET ind-mis-carl.shalltry.com/CarlcareClient/electronic-card/check-extended_warranty-web?imei=
sign: md5(SALT + imei)
The sign header is mandatory (without it: code 10022 "Sorry, web sign is error"). The salt
is in the site's own bundle, _nuxt/085537c.js module 688, along with the base url; re-read
that bundle if it ever stops working. status 3 = activated and carries activeTime, status
2 = device known but not activated yet and activeTime is null.
It is also the semantically right source. The DCR portal serves an INVENTORY report and
the old SAP feed served a TERTIARY SALES report, whereas activeTime is the date the
handset was actually activated -- which is what fofo.activated_imei is meant to hold and
what tertiary payout is computed on.
Measured against hadb1 before writing any of this: tecno 355463920708766 -> 2026-08-28 and
itel 359207322028000 -> 2026-08-30, both exact matches to rows we already had. A read-only
trial of 50 itel imeis answered 50/50 with no errors and no sign rejections, agreed with
our catalog brand on all 50, and returned a date for 18 -- every one of those 18 falling
AFTER our own billing date, 3 to 1,111 days, median ~220.
SHAPE
Pool queries and saveActivation semantics are the vivo ones, so the two read alike, and no
DAO change was needed: the pending queries are already brand-generic. The far end is far
cheaper than vivo's, one signed GET per imei, so there is no captcha service, no cookie
store, no session seeding and no verdict reporting.
50 imeis every 5 minutes = 14,400 lookups a day against ~10,400 pending itel, so the whole
pool is covered daily with headroom. The pool reaches back to 2021, so the ticks are
themselves the backfill of the nov-2024 blackout; there is no one-off to run.
Two things worth knowing before changing it:
- The batch is drawn ROUND-ROBIN across each (brand, channel) queue, not by concatenating
them. A full daily pass can concatenate freely because it walks to the end, but a
50-at-a-time tick cannot: itel is ~1,550 secondary against ~8,851 tertiary, so the head
of a concatenated list is ~31 straight ticks of pure secondary before one tertiary imei
is asked about. The first trial batch was 100% secondary for exactly that reason.
- Every outcome stamps the row, failures included. This is the one deliberate departure
from vivo, which leaves a failure unrecorded so it retries next pass -- safe there
because the next pass is tomorrow. On a 5-minute cadence it is not: an unstamped imei is
due again in five minutes, the query keeps handing back the same 50 rows, the batch never
advances past them, and the endpoint is asked the same questions twelve times an hour for
as long as it keeps failing. See the runaway documented on oppoRealmeImeiActivation.
BRANDS is itel alone to start. That is a rollout order, not a limit of the endpoint: itel
is the brand we can CHECK, because the DCR pull independently produces itel dates to
reconcile against, where tecno has nothing. Add "Tecno" once a day's rows agree. Both feeds
may write itel meanwhile with no coordination -- the pool query only returns imeis whose
activationTimestamp is still null, so whatever one fills has left the other's pool.
Scheduled tick plus a --checkCarlcareImeiActivation flag, both wired. |
|
| 37480 |
22 h 12 m |
amit |
/trunk/ |
mail: correct a wrong claim in r37479 -- the Google app password IS valid
r37479 stated googleMailSender's app password was rejected. That was wrong. The
test behind it resolved smtp.gmail.com over IPv6; repeating it over IPv4 with
the same credential gives AUTH OK on both 465 and 587.
The real fault is not the credential and not the bean config, both of which are
correct. SMTP from this host works over IPv4 only:
smtp.gmail.com IPv4 -> AUTH OK IPv6 -> 535 5.7.8 Username and Password not accepted
smtp-relay IPv4 -> 250 MAIL FROM IPv6 -> 550 5.7.1 Invalid credentials for relay
The JVM prefers IPv4, which is the only reason mail leaves this box at all.
Anything that prefers IPv6 fails on both paths. |
|
| 37479 |
22 h 18 m |
amit |
/trunk/ |
mail: make the relay the default sender, not the Google identity
Correcting r37474-37476. Those made googleMailSender @Primary on the assumption
its credentials worked. Tested against the live servers from the prod host:
googleMailSender 535 5.7.8 Username and Password not accepted (BadCredentials)
-- the app password in the source is no longer valid
relay over IPv4 250 OK for MAIL FROM:<noreply@smartdukaan.com>
relay over IPv6 550 5.7.1 Invalid credentials for relay
So promoting google would have replaced one broken default with another. The
relay is what actually delivers today and it becomes 'mailSender'. It does not
authenticate -- Workspace authorises it by allowlisted source IP -- so sending
as noreply@ is legitimate there and AuthenticatedIdentityMailSender correctly
leaves it alone.
googleMailSender stays available by qualifier. Point the default back at it once
a valid app password is issued for sdtech@smartdukaan.com.
Also noted in the javadoc: the relay allowlist covers the IPv4 address only, so
anything that prefers IPv6 will be refused. |
|
| 37474 |
22 h 56 m |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
mail: delete the dead SendGrid bean from cron, make the authenticated identity primary
The @Primary bean named 'mailSender' was SendGrid, so every unqualified
JavaMailSender injection got it -- and SendGrid rejects with 535, which is what
was failing 10 report/notification cron jobs (dailyTrackingReport,
monthlyTargetForPartner, sendFeebackSalesAndRbm and others). 420 occurrences in
one log.
googleMailSender now answers to both 'googleMailSender' and 'mailSender' and is
@Primary, so those injections resolve to a sender that works and whose From is
rewritten to the authenticated identity. No call site changes. |
|
| 37472 |
23 h 59 m |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
IMEI activation: one snapshotted daily pass per brand, on one thread, with per-brand metrics
Four @Scheduled jobs every 5 minutes become two daily passes. Oppo and realme
share one thread and alternate in 25-imei chunks, so exactly one ChromeDriver is
alive at a time instead of four; vivo keeps its own thread since it is direct
HTTP and does not contend for a browser.
The pass snapshots its pool before any browser starts and walks that list to the
end. It never re-queries, and that is the actual fix. A failed lookup never
reaches dateMap.put, so no row is written, so createTimestamp is not bumped, so
the imei was eligible again on the next tick five minutes later. Measured 29-Aug:
realme issued 4,524 requests against 1,004 distinct imeis -- 4.5 asks each, 78%
of the day's budget spent re-asking -- while oppo, which rarely fails, sat at
1.03. More requests hardened the block, which caused more failures. A pass bounds
that: a failure costs one retry tomorrow, never one in five minutes.
This supersedes the r37447/r37448/r37449 argument about driver count, which was
about the wrong variable. That argument blamed realme's collapse on CPU
contention pushing the captcha render past the element waits. The logs do not
support it: on 29-Aug oppo took ZERO canvas timeouts across all 24 hours on the
same box, same six cores, same driver count, same captcha vendor, load average
0.9 -- including the 15:00-23:00 window in which realme solved nothing at all.
Realme's own canvas wait is 15s against oppo's 8s, so the longer wait is the one
expiring. What realme's timeout rate tracks is its own daily request volume, and
it resets at midnight: 920/day -> 0.3%, 3,467/day -> 28%, 4,524/day -> 75%. That
is realme.com declining to serve the widget.
DAYS=0 is deliberate and is not an off-by-one: the pool filter is
createTimestamp < now().atStartOfDay().minusDays(DAYS), so DAYS=1 measures
against yesterday midnight and silently yields a two-day cadence, which is what
oppo and realme were running.
Sizing measured on prod for a midnight start: oppo 4,133 and realme 2,118 imeis,
11.7h + 8.4h = 20.1 hours of a single thread. It fits with no slack; if the
'pass finished' counts come in short of 'pass starting', the lever is DAYS=1
rather than a second thread.
Observability: ImeiActivationGauges publishes the funnel per brand on
/actuator/prometheus, which alloy already scrapes on this host -- due, churned,
captcha_shown, captcha_solved, answered, dates_found, errors, run_seconds and
last_finish_epoch. Each stage fails differently and says what broke. Rates are
left to PromQL. The stage that matters for health is answered: churned>0 with
answered==0 is precisely the shape of both silent outages this year (oppo wrote
nothing for a week; the vivo captcha solver was dead for 46 days). dates_found is
deliberately NOT a health signal -- when the multi-year backlog drained at the
end of August, yield fell from ~100% to 2-3% on the same day across all three
brands with nothing broken.
Nagios cleanup: the Nagios server and every NRPE daemon are gone, so
WriteToPropertiesFile and the commented-out blocks that fed
nagios-cron.properties are deleted, and NagiosMonitorTasks is renamed
BalanceMonitorTasks for the transport it actually uses. Noted there that nothing
calls it -- there is no @Scheduled entry and no other caller -- which is why both
balance gauges have always read -1. |
|
| 37470 |
1 d 0 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Remove dead Itel/Tecno SAP activation services
Both services call SAP OData endpoints that Transsion decommissioned on 2024-11-20:
ItelImeiActivationService -> cms.itel-mobile.com:8099/.../ZTERTIARY_SALES_REPRT_SRV
TecnoImeiActivation -> cms.tecno-mobile.com:8099/.../ZTERTIARY_SALES_REPRT_SRV
Both hosts now refuse connections outright (verified from the prod app server).
That decommissioning is what stopped Itel and Tecno activation ingest within the
same hour on 2024-11-20; everything since has arrived via manual CSV upload.
Neither service has produced a row in 21 months and neither can again.
Removed:
- both service classes
- ScheduledTasks.checkItelImeiActivation / .checkTecnoImeiActivation, their
@Autowired fields and imports (the only callers)
- the --checkItelImeiActivation / --checkTecnoImeiActivation startup args
Unaffected: ItelImeiActivationNewService and checkItelImeiActivationNew, which
target the current imwav portal and were fixed in r37467. |
|
| 37467 |
1 d 4 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/itelImeiActivation/ |
Fix Itel/Tecno activation pull: live session tokens, correct credentials, per-account isolation
The imwav pull has never worked since it replaced the SAP feed. Three defects:
- Login response was discarded and the listing call sent p-auth/utoken values
captured by hand in Jul-2023, so every request went out with an expired session.
Now takes utoken/urToken from the login response.
- Itel (IN404338I) carried a stale password (Dukaan@1234), which is why its pull
started failing Feb-2026. Corrected, and the two missing Tecno branch accounts
(Noida IN411226T, Lucknow IN404339T) added - the code previously knew only
IN404338T, which has no enterprise scope and returns nothing.
- Existing rows were looked up by the activation date string instead of the IMEI.
serial_number is the primary key, so the lookup always missed and re-persisted a
row that already existed, colliding on duplicate key on any re-pull.
Also pulls each account independently so one dead login cannot abort the remaining
accounts, and fails loudly when the portal rejects a login or the report call
instead of proceeding into a confusing downstream error. |
|
| 37465 |
1 d 4 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Exclude aged Apple stock from investment when suggesting partner credit limits
Apple handsets a partner has held more than 30 days from GRN no longer count
toward the investment that drives the suggested limit. Applied in
getSuggestedAmount via getCreditableInvestment, covering the SDDIRECT,
hundred-percent and SIDBI branches. The aged amount is logged alongside each
limit change so a drop can be traced.
Scoped to the limit calculation only -- getTotalInvestment() is untouched, so
checkout payment options, the investment-OK gates and partner-facing screens
see the same stock value as before.
Measured on prod at time of commit: 580 units / ~3.94 Cr across 88 partners. |
|
| 37459 |
3 d 1 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Motorola IMEI activation: secondary + tertiary in one browser session
Mirrors the oppo/realme combined jobs. MotorolaImeiActivationService drives
CheckMotorolaWarrantyTask, with MotorolaChallengeSolver for the challenge.
Cadence comes from the pool query, which defers an imei for `days` after each
attempt (saveActivation bumps createTimestamp even when no date came back), so
days=2 retries everything every two days. Pending pool measured 1,534
(1,182 secondary + 352 tertiary); at ~10-14s/imei, 60 per invocation is about
12 minutes of driver time and clearing the pool inside 48h needs roughly 26
invocations, i.e. an OS cron entry every 90 minutes.
Do NOT schedule it inside the oppo/realme window: each driver tree costs
~850MB and this box has been OOM-killed twice with tomcat the victim, so peak
concurrent drivers is the number that matters. |
|
| 37453 |
3 d 8 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/ |
Build googleMailSender as AuthenticatedIdentityMailSender
The bean authenticates as sdtech@smartdukaan.com, so Google Workspace only permits
that address in From. Callers hardcode noreply@, which Gmail refuses with 535
'Authenticated user is not authorized to send mail', silently dropping every alert
sent through it. The sender now rewrites From to its own username at send time.
See profitmandi-common r37451. |
|
| 37450 |
3 d 8 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Oppo: find only the hole, and let a low-contrast hole be found at all
60% of oppo attempts never got past detection -- getMatCircles2 returned null and
the attempt was wasted before any aiming happened. That is a bigger loss than
everything the scheduling work addressed put together.
Two causes, both visible once real puzzles are captured and looked at.
1. param1 is the Canny HIGH threshold, and it was 100. Edges weaker than that are
discarded before circle finding begins. Measured hole-vs-background contrast on
four live puzzles: 98, 103, 28, 117. The 28 -- a pale lilac background -- cannot
produce an edge at 100, so no circle exists to find. Dropped to 50.
2. It insisted on TWO circles whose radii matched within 3px, and returned null
otherwise. So a puzzle where the hole was located perfectly still failed if the
ring was missed, or if the two radii differed by 4px.
The second is the sillier one: the ring is a DOM element,
dx_captcha_basic_sub-slider_, whose exact position aim() already reads via
pieceCentreX(). Detection was re-finding something known exactly, and then
throwing away a good hole because it could not confirm it. Now the ring position
is passed in and hough only has to find one thing.
Lowering param1 without the pairing check would let textured backgrounds (the
sand images especially) supply spurious circles with nothing to reject them, so
the candidate is verified as an actual hole: its core must be at least 15
luminance below the frame mean. Real holes measure 28-117 below, so 15 rejects
noise with margin.
Validated against live puzzles before committing:
ring is at rel 42px on every sample; hole lands at 113-194px, separation
71-152px -- so RING_EXCLUSION_PX = 30 never rejects a real hole
hole contrast 28-117 against MIN_HOLE_DARKNESS = 15
Not validated locally: the hough call itself. opencv 3.4.2-0 ships no osx/arm64
native and every JDK on this machine is arm64, so the detection path cannot run
here. param1 = 50 is reasoned from the measured contrasts, not measured. If the
detection rate does not move, 40 is the next value to try -- the darkness check is
what makes going lower safe.
Watch "Detected N circle(s) but all sat on the ring" and "Rejecting circle at Npx"
in the log: the first means hough found nothing but the ring, the second means the
darkness check is doing its job. |
|
| 37449 |
3 d 19 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Realme unmerged too: four selenium jobs, one per pool per brand
r37447 merged secondary and tertiary per brand to halve concurrent drivers.
r37448 reverted that for oppo, because serialising cost it more per imei than a
second browser did. This reverts it for realme as well, which ends the merge
experiment entirely.
What the merge actually cost realme is only visible now that oppo is parallel
again: realme's per-imei went 14.2s -> 29s. It did not change behaviour -- it is
simply competing with oppo's two browsers for CPU. At 29s its merged ceiling is
86400/29 = 2,979/day, just under the 3,054/day the daily re-check needs, so no
batch size could have closed the gap. Two parallel pools restore the ~3,900/day
it managed historically at four drivers.
Measured before this change (25 min window):
brand needed/day throughput note
Oppo 8,838 7,661 parallel revert worked, +122%
Realme 3,054 1,843 merged and CPU-starved
Vivo 10,082 16,128 surplus, cannot transfer to another brand
Sizes unchanged: oppo 25 per pool, realme 12 per pool, vivo 50+10.
Honest accounting of the merge: it was my idea, sized on per-brand arithmetic that
ignored contention between brands, and it is now fully reverted. What survives
from that line of work is the part that actually paid -- reaping orphaned browsers
(~1.8GB), the per-brand retry caps, and per-brand maxResults. Peak drivers are
back to 4, which is where they started.
Watch for contention: four browsers is the configuration that produced the 29s
figure for realme in the first place, so oppo may slow from its current 19.7s.
Re-measure both before tuning sizes again. |
|
| 37448 |
4 d 3 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Daily re-check for all brands; Oppo back to parallel pools
Two changes.
1. Re-check window 4 days (secondary) and 2 days (tertiary) -> 1 day everywhere.
Daily demand becomes the full universe rather than a fraction of it:
Oppo 4,798 + 4,040 = 8,838/day
Vivo 9,407 + 675 = 10,082/day (doing 15,345 -- fine)
Realme 1,973 + 1,081 = 3,054/day
2. Oppo's two pools run in PARALLEL again, reverting the merge in r37447 for that
brand only. Realme stays merged.
The merge was a straight trade of throughput for memory and oppo could not
afford it. Measured over 32 minutes and again over an hour the next morning:
3,555 then 3,456/day against 5,280 before merging. Batch cadence settled at a
very regular ~15.5 min per cycle, so a 30-imei merged batch takes ~10.5 min =
~21s/imei, against the 10.2s it managed unmerged. At 21s the ceiling is
86400/21 = 4,114/day even with zero idle, so no batch size and no shorter
fixedDelay could have reached 8,838. Serialising simply costs more per imei
here than running two browsers does.
Realme keeps the merge: it needs 3,054/day and delivers 2,952 merged, so a
small size bump covers it without a second browser.
Sizes: oppo 25 per pool (2 jobs in parallel), realme 12+12 merged, vivo 50+10
unchanged. Vivo has already cleared its entire secondary backlog -- the pool
reads 0 and both lists come back empty -- which is what the batch of 50 was for.
Cost: oppo goes back to two concurrent drivers, so the fleet is 3 rather than 2,
roughly +700MB. Acceptable against the ~2GB freed today by reaping orphaned
browsers and capping retries, but it is the reason realme was left merged.
Sizes are a starting point, not a final answer: oppo's per-imei time differs
markedly between merged and parallel modes, so re-measure before tuning further. |
|
| 37447 |
4 d 19 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Selenium: one browser per brand instead of one per pool
Oppo and Realme each ran secondary and tertiary as separate @Scheduled jobs, so
each brand opened two ChromeDrivers at once and the fleet ran four. Each driver
tree costs roughly 850MB. This box co-hosts a 9.4GB tomcat with ~3GB available
and has been OOM-killed twice this month -- tomcat was the victim both times, so
peak concurrency is the thing that matters here.
Combined into one job per brand. Nothing downstream changes: the two pools differ
only in which named query fills them, and both already funnel into the same
updateActivationDate -> checkWarranty -> saveActivation path. They are disjoint by
construction (secondary excludes anything with a FofoLineItem, tertiary is
FofoLineItem-based); distinct() is insurance, not a fix for a known overlap.
Sizing matters, because merging SERIALISES work that used to run in parallel and
keeping the old batch sizes would quietly cost throughput. Measured post-cap at
10.2s/imei (oppo, down from 14.6 after r37445) and 14.2s/imei (realme), solving
M * 86400 / (300 + M*t):
oppo 2 parallel jobs x10 = 4,299/day -> merged 15+15 = 4,277/day (parity)
realme 2 parallel jobs x10 = 3,910/day -> merged 10+10 = 2,959/day (-24%)
Oppo is sized to hold parity because it is already short of its 4,798/day need.
Realme is left at 20 -- it needs 2,243/day, so it can absorb the dip in exchange
for shorter batches and a shorter-lived browser.
What this saves and does not save: total driver-SECONDS are roughly unchanged,
which is the point of resizing. PEAK concurrent drivers halves from 4 to 2.
Also skips starting a browser at all when both pools come back empty -- currently
never true, but it costs nothing and a browser launched to do nothing is pure
waste on this box.
checkOppoImeiStatus/Tertiary and the realme equivalents are left in place for
manual invocation; they are simply no longer scheduled. |
|
| 37446 |
4 d 19 h |
amit |
/trunk/ |
Per-brand batch sizes, and stop chrome forking a GPU process it cannot use
maxResults was hardcoded in the shared repository methods, so Oppo and Vivo were
forced to the same secondary batch (10) and all three to the same tertiary (10).
It is now a parameter, set per brand at the call site.
Sizing is arithmetic, from measured IN-BATCH per-imei time. Solving
M * 86400 / (300 + M*t) = needed/day:
brand needed/day t M required set to
Vivo 9,407 0.8s 36 50 clears, ~12,700/day
Realme 1,973 13.4s 10 10 was 5 = ~1,177/day, short
Oppo 4,798 14.6s 88 10 HELD, see below
Correcting an earlier measurement of mine: I reported Vivo at 13.6s per imei and
concluded its backlog could not be cleared. That averaged across the ~300s idle
gaps BETWEEN batches. In-batch it is 0.8s -- Vivo is 17x faster than I said, is
idle ~97% of the time, and 50 clears its pool comfortably. There is no wait in
the Vivo path; it is simply fast.
Oppo is deliberately NOT raised. At 14.6s it would need M=88, which means
20-minute batches and near-permanent chrome sessions. But that 14.6s predates the
retry cap (r37445), which cuts exhausted imeis from 20 attempts to 7 and should
drop it sharply. Re-measure before sizing Oppo, rather than guessing high on a
box with 3GB free.
Also: --disable-gpu, --disable-dev-shm-usage, --disable-software-rasterizer on
both selenium tasks. Headless needs no GPU yet chrome forks a gpu-process per
browser -- 6 were alive across the fleet, pure overhead. No behaviour change.
Batch size does not raise peak concurrency (fixedDelay means one batch per job at
a time, so never more than 4 drivers). It raises DUTY CYCLE, which converts
chrome's footprint from intermittent to sustained. That matters here: tomcat is
9.4GB, available is ~3GB, and the two OOM kills this month both took tomcat.
Cron-only deploy. The dao signature change has no callers outside cron. |
|
| 37445 |
4 d 21 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Oppo/Realme: cap captcha retries per tick at 7 and 12, measured per brand
Most captcha work goes to imeis that never succeed. Measured over the last
window: oppo 6,245 attempts for 308 successes, realme 13,842 for 399. The bulk
is imeis grinding all 20 attempts in one already-refused session and getting
nothing -- oppo 279 of them, realme 610.
The imei is NOT abandoned. A captcha failure is a technical failure, not an
answer, so it stays pending and the next tick retries it in ~5 minutes with a
fresh page and session -- which the data suggests beats continuing in a session
the widget has refused seven times. This caps grinding, not retrying.
Per-brand caps, because the distributions genuinely differ:
OPPO cap 7 97.7% of successes kept, 58.4% of work saved (3,648 attempts)
Pre-glide, 33% of oppo's successes came from attempts 8-20 and 20 was the
right number. Post-glide (r37440) that is 2%: the drag lands first or second
try. cap 5 keeps 95.5%/saves 67.7%, cap 10 keeps 99.0%/saves 44.8%.
REALME cap 12 96.0% kept, 35.7% saved (4,937 attempts)
NOT 7. Realme has no glide, so its successes still spread to attempt 10+ and
a cap of 7 would cost it 16.5%. It also burns more than twice oppo's work, so
the gentler cap still saves more in absolute terms.
Combined: 8,585 of 20,087 attempts saved, ~43% less browser work, for ~3% fewer
successes per tick -- and those imeis come back next tick anyway. Less Chrome
work matters on this box: it co-hosts a 9.4GB tomcat, has 3.5GB free and has been
OOM-killed twice this month.
The two caps are independent constants and must not be synced. If glide is ported
to realme, expect its curve to shift left as oppo's did and its cap can drop to
~7 -- but measure it, do not assume it.
Windows are ~2h (oppo's only ~30 min post-glide), so the right numbers may drift.
Worth re-reading the attempt histogram after a full day. |
|
| 37441 |
4 d 23 h |
ranu |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
bi access to gaurav mathur |
|
| 37440 |
5 d 2 h |
amit |
/trunk/profitmandi-cron/src/main/java/com/smartdukaan/cron/scheduled/ |
Oppo: slide the drag in eased sub-moves instead of jumping
Aim is solved and it did not finish the job: r37438 lands the ring 0-1px from the
hole on 103 of 104 drags and oppo still converts ~15%, with POSITION_MISMATCH on
the rest. So the widget is refusing drags that end in the right place.
Dingxiang scores trajectory and velocity as well as final position and returns the
same POSITION_MISMATCH for both, so our verdict cannot separate "wrong place" from
"wrong-looking movement". The circumstantial case for the latter:
oppo 0.91s hold, 2-3 discrete jumps -> 15%
realme 1.93s hold, slower and more -> 54-74%
and r37438 made oppo FASTER by removing the per-step screenshots, moving it further
from realme's profile rather than closer.
glide() traverses each leg as up to 18 small sub-moves on an ease-out curve
(1-(1-t)^2, quick then settling) with a 45ms pause between, so the pointer
describes a continuous decelerating path instead of teleporting.
Nothing about the DISTANCE changes -- it is still exactly what the derived ratio
says, and verified to land on it precisely: a 116px leg becomes
[16,15,13,13,11,10,9,8,6,6,4,3,1,1] and sums to 116. Negative legs work the same
way for overshoot correction.
To be explicit, since this is the thing not to get wrong: the ratio remains fully
derived. It starts at zero and comes only from measured movement, per attempt and
per step. Splitting a leg into sub-moves gives MORE measured legs, not fewer, so
the estimate sharpens rather than being replaced by an assumption. No constant is
introduced -- MAX_GLIDE_STEPS, GLIDE_PX_PER_STEP and GLIDE_PAUSE_MS describe the
path, not the relationship between mouse px and ring px.
Expected hold ~1.7s against realme's 1.93s. Oppo only.
This is a hypothesis test, not a known fix. POSITION_MISMATCH rate answers it in
one tick; if it does not move, drag dynamics is eliminated and the remaining
suspect is Hough picking the wrong circle pair on oppo's imagery, which needs
visual evidence rather than another code change. |
|