Modem stopped working?

and I too ((((

I have exact the same problem

I have the same issue

Running into the same issue on my device. Went away on vacation for two weeks, came back, and the modem doesn't seem to boot. I've got it sitting on my desk now with USB for testing, but it's just stuck in the power cycle loop. Not seeing D5 on the module ever light up. Upon boot, this is what I see:

I (18863) webserver: HTTP POST /shell

I (18873) webserver: HandleLogin: 'admin' logged in, sid 36b6e7088a3282f7

I (18903) webserver: HTTP GET /menu

I (18913) webserver: HTTP GET /shell

I (21243) simcom: State timeout, transition to 2

I (21243) simcom: State: Enter PoweringOn state

I (21243) simcom: Power Cycle

V (22243) v-nissanleaf: Range: ideal=0 km, est=0 km, full=0 km

D (22243) v-nissanleaf: Poll state: 0

V (31243) v-nissanleaf: Range: ideal=0 km, est=0 km, full=0 km

D (31243) v-nissanleaf: Poll state: 0

I (41243) simcom: State timeout, transition to 2

I (41243) simcom: State: Enter PoweringOn state

I (41243) simcom: Power Cycle

V (42243) v-nissanleaf: Range: ideal=0 km, est=0 km, full=0 km

D (42243) v-nissanleaf: Poll state: 0

V (51243) v-nissanleaf: Range: ideal=0 km, est=0 km, full=0 km

D (51243) v-nissanleaf: Poll state: 0

I (61243) simcom: State timeout, transition to 2

I (61243) simcom: State: Enter PoweringOn state

I (61243) simcom: Power Cycle

V (62243) v-nissanleaf: Range: ideal=0 km, est=0 km, full=0 km

D (62243) v-nissanleaf: Poll state: 0

V (71243) v-nissanleaf: Range: ideal=0 km, est=0 km, full=0 km

D (71243) v-nissanleaf: Poll state: 0

I (71243) ovms-server-v2: Send MP-0 S0,K,0,0.00,stopped,standard,0,0,0,0,0,0,0,21,0,0,0,0,0.00,0,0,0,0,-1,0,0,0,0,0,0,0,0.00,0.00,0,0.00,100.00,0.00,0.0

I (71243) ovms-server-v2: Send MP-0 D0,0,5,0,0,0,0,0,0,70,0,0,1,1,0,0,12.6,0,0,0,0

I (71253) ovms-server-v2: Send MP-0 F3.2.018/ota_0/main (build idf v3.3.4-848-g1ff5e24b1 Nov 22 2021 09:37:27),,30,1,NL,Alex's Galaxy S9,-1,-1

I (81243) simcom: State timeout, transition to 2

I (81243) simcom: State: Enter PoweringOn state

I (81243) simcom: Power Cycle

V (82243) v-nissanleaf: Range: ideal=0 km, est=0 km, full=0 km

D (82243) v-nissanleaf: Poll state: 0

V (91243) v-nissanleaf: Range: ideal=0 km, est=0 km, full=0 km

D (91243) v-nissanleaf: Poll state: 0

I (101243) simcom: State timeout, transition to 2

I (101243) simcom: State: Enter PoweringOn state

I (101243) simcom: Power Cycle

V (102243) v-nissanleaf: Range: ideal=0 km, est=0 km, full=0 km

D (102243) v-nissanleaf: Poll state: 0

V (111243) v-nissanleaf: Range: ideal=0 km, est=0 km, full=0 km

D (111243) v-nissanleaf: Poll state: 0

I (121243) simcom: State timeout, transition to 2

I (121243) simcom: State: Enter PoweringOn state

I (121243) simcom: Power Cycle

V (122243) v-nissanleaf: Range: ideal=0 km, est=0 km, full=0 km

D (122243) v-nissanleaf: Poll state: 0

V (131243) v-nissanleaf: Range: ideal=0 km, est=0 km, full=0 km

D (131243) v-nissanleaf: Poll state: 0

I (132243) ovms-server-v2: Send MP-0 S0,K,0,0.00,stopped,standard,0,0,0,0,0,0,0,21,0,0,0,0,0.00,0,0,0,0,-1,0,0,0,0,0,0,0,0.00,0.00,0,0.00,100.00,0.00,0.0

I (132243) ovms-server-v2: Send MP-0 D0,0,5,0,0,0,0,0,0,131,0,0,0,0,0,0,12.6,0,0,0,0

I (132243) ovms-server-v2: Send MP-0 F3.2.018/ota_0/main (build idf v3.3.4-848-g1ff5e24b1 Nov 22 2021 09:37:27),,31,1,NL,Alex's Galaxy S9,-1,-1

I (141243) simcom: State timeout, transition to 2

I (141243) simcom: State: Enter PoweringOn state

I (141243) simcom: Power Cycle

V (142243) v-nissanleaf: Range: ideal=0 km, est=0 km, full=0 km

D (142243) v-nissanleaf: Poll state: 0

Hopefully will have some time to look into this over the coming days, but I'm worried that the modem board failed...

I've done a little bit more testing on this, but haven't determined the source of the problem. I currently have the OVMS module plugged into a power bar with dedicated 2.4 A USB outlets. A couple of notes (all reference designators on modem board):

• I'm seeing 3.308 V on JP1.1 (+3V3) as I would expect
• I'm seeing 4.90 V on JP1.22 (USB_5V) which is in line with the 4.75 V to 5.25 V range in the USB specification
• I'm seeing 4.60 V on JP1.24 (EXT_PWR) which is a good indication that the modem is indeed not turning on
  • EXT_PWR is generated by ORing together the USB_5V and EXT_12V rails through STPS2L60 diodes
  • As per Figure 10 in the datasheet, a forward voltage of ~0.3 V indicates that the forward current is very small (less than 100 mA)
• I'm seeing 0.585 V on the VBAT rail (measured at C15 and U2.39,40)
  • VBAT is generated by the AOZ1280CI switching regulator U1
  • As per the datasheet, Vo = 0.8*(1+R1/R2), where in our case R1 = 49k9 and R2 = 12k5, so Vo = 3.9936 V or ~4.0V
  • If I take the modem board off of the main board, power it off USB, and then short USB_5V to EXT_PWR, I still only see 0.585 V on the VBAT rail
  • I think this is a good indication that something might be wrong with the hardware...
• JP1.15 (MDM_EN) starts at 0.0574 V and jumps to >3.8 V every 20 seconds (I don't have a peak hold function on my multimeter) which is what I'd expect from this pin
  • The MDM_EN signal is used to toggle the modem on and off by pulling U2.3 (PWRKEY/POWER_ON) low greater than 50 ms
  • The MDM_EN signal is generated by an I/O expander on the main board (U9.3, signal MEM_EN which is a typo; should be MDM_EN)
• Using the MDM_EN signal only, U2.3 (PWR_KEY) starts at ~0.2556 V and transitions to 0 V when MDM_EN goes high
  • According to the SIM5360 hardware design guide section 3.4.1, the "PWRKEY" pin is pulled up to 1.8 V internally through a 200k0 resistor

I might try and bring the module into work to see if I can get any oscilloscope captures

There's plenty of stock of AP3211KTR-G1 at Digi-Key currently. Same pinout, similar package dimensions, similar specs and feedback resistor setup. I don't like the smaller input voltage window (4.5 V to 18 V vs. the AO1280CI's at 3 V to 26 V); how high can EXT_PWR go?

I might get a couple of these and try them out on my board.

Thanks, I'll submit a support ticket then. Where are you purchasing the parts from usually?

I used AP3211KTR-G1 as a drop-in replacement (i.e. no other PCB or component changes required), and now the NETLIGHT is blinking and it's connected to the network! Who knows if this will work long term, but for now it seems to be working fine.

My OVMS v3 (new from Medlock just 1-1/2 months ago) worked for about a month, then dropped off cellular. In diagnosing it, there's a data session open with Hologram, but 0 data on it. The modem status page says it's "denied registration". I've tried everything short of opening up the box and/or throwing it out the window.

Submitted a support ticket with Hologram, and they kinda just pointed the finger back at me: their side looks fine. But why/how could it be "Denied registration" if it's not their fault?

Looking over this thread, it seems plausible that the modem design isn't quite perfect, and maybe this could be affecting me as well. You might notice I said, I stopped short of opening the box -- I really didn't want to get nitty gritty into this thing (though I am fairly capable - just not something I really wanted to dig into). I guess, though, that may be the only way forward - possibly replacing a defective voltage regulator?

Guess I might start with analyzing the voltages and seeing if I need to replace that buck controller as well...

First thing I'd check is D5 on the modem board. If the modem chip is coming up properly, D5 will light up and blink with network activity after being plugged in over the microUSB port and shorting USB_5V to EXT_PWR (I did this on JP1) on that board. If it doesn't come up at all, then it would be good to check the VBAT rail voltage.

@markwj, when you said you've seen the bucks failing recently, does that include modem boards that were just purchased, or those that had been in use for a long time?

I'm still having no luck, and the OVMS now just sits in a sad state on my couch.

I can't find any hardware issues. I get abour 4.0v at VBAT, steady, when powered by USB. I _deeply_ dislike that I have to bend/short two pins on the modem board to get its USB power to come up -- and then I have to hold down the unlabeled button to get the power to start, as well. All seems basically undocumented - had to learn that through schematics. (silkscreen is free! haha)

Using AT commands with the board removed, I ran through everything, and I found that it had a forbidden list entry - something-or-other FPLMS or whatnot. Great, but I clear that, no improvement.

The closest I was able to get was to swap SIMs and insert my trusty old Ting Mobile SIM into the thing. Then, it quickly reaches "RegisteredHome" mode, but PPP connections fail and it never gets online (and yes, I set the APN -- and I even used AT commands to verify it's set properly in the modem - for Ting it should be "wholesale").

So, with the Hologram SIM, all I can get is "Registration Denied". With the Ting SIM, I can get "RegisteredHome", but no PPP connection.

With a whole slew of diagnostics, all Hologram has given me (admittedly, more than I'd expect for the $2/mo or so that I'm paying them) has been "it's your device's fault". Seems to be a dead end - from their side, everything is clean.

I notice that D5 blinks at different periods for different operational modes. Fast blink - it's searching. Slow blink - it's registered. Solid on = it's offline/stuck. I can do "AT+CFUN=0" and it takes the modem totally offline - "AT+CFUN=1" and it's back to life, for example.

It's massively frustrating that there doesn't seem to be an AT command console in the OVMS system -- you can send blind AT commands, but no result is returned by those commands. Means that you have to tear apart the hardware in order to uncouple the OVMS<->modem, in order to send AT commands -- and you can't send any AT commands to get status from the modem while it's connected. Surely this isn't ... normal?

Good to hear that it turns on at least. Out of curiosity, what network should Hologram be connected to in your area? Is there a possibility that there's poor coverage on that network?

Also, when Hologram points to your device as the issue, I assume that they're never able to actually see the device come up on their side (since it doesn't register on a network), so they think that the device isn't connecting properly to whatever network they're using.

No, from their side, they see a data session, the connection is good, everything seems like it should work. That's what's so confusing - all the low hanging fruit appears to be picked. Every time it wants to come up, it finds the network, and Hologram sees an active data session - it's just 0-bytes every time, connected for hours/days at a time, no traffic. As far as Hologram is concerned, the device is there, fully functional, ready to serve traffic, good signal, everything - it's just that nothing happens. It's like dialing a phone number, and both parties just sit there listening to each other breathe -- in perfect clarity.

In my area, it comes up as T-Mobile - same as my personal phone, which of course is working fine :)

That then leads to my experiment with Ting - where if I put the Ting SIM in there, it comes up as registered, but PPP doesn't connect (let me rephrase that: the logs basically consist of saying that "PPP disconnected", then it tries restarting, and a couple more lines saying "PPP disconnected". No real error is really shown!). The cell system of OVMS is pretty opaque - is there anything we could even do to diagnose that?

Here is the loop I get stuck-in under Ting:

 (170084) cellular: State: Enter NetMode state
I (170084) gsm-ppp: Initialising...
I (170204) cellular: PPP Connection disconnected
I (170204) cellular: PPP Connection disconnected
W (171084) cellular: Lost network connection (+PPP disconnect in NetMode)
I (171084) cellular: State: Enter NetLoss state
I (171084) gsm-ppp: Shutting down (hard)...
I (171084) gsm-ppp: PPP is shutdown
I (181084) cellular: State: Enter NetWait state

Over and over. Eventually the cell LED stops flashing.

Going to try "as factory a reset as I can figure out how to do" on both the modem and the OVMS next. BTW, I'm on firmware 3.3.002, for what it's worth...

...

OK, so I did it before I sent this. Factory reset OVMS first, switched to modem USB, sent "AT&F" (factory reset) then "AT+CFUN=6" to reset/reboot, verified settings were wiped. Plugged it all back together, and it was factory fresh.

Same. Exact. Story. With the Ting SIM. PPP infinite loop. Hologram "DeniedRegistration".

I (541154) cellular: Network Registration status: DeniedRegistration
I (541164) cellular: Network Provider is: T-Mobile Hologram
I (543544) cellular: Signal Quality is: 26 (-61 dBm)

I'm completely f*cking stumped.

No idea about your issues, but to debug cellular, set the log level for "cellular" to debug or verbose.

This will also enable output of AT command responses to the log (though some like "OK" will be filtered).

OVMS# log level debug cellular
Logging level for cellular set to debug
OVMS# cellular cmd AT+COPS?
MODEM command has been sent.
D (305848) cellular: mux-tx-cmd #4: AT+COPS?
D (305868) cellular: mux-rx-line #4: +COPS: 0,0,"vodafone.de Hologram",7

Hello,

The cell phone works until the restart, if I disconnect and reconnect the ovms I have to connect to wifi to activate the rear cell phone, but there is a solution?

Hello,

The cell phone works until the restart, if I disconnect and reconnect the ovms I have to connect to wifi to activate the rear cell phone, but there is a solution?

We have identified one cause of the 'Denied Registation' issue. Some cellular providers can accept registration at LTE level, but deny registration at GSM/GPRS level.This is interpreted as a 'Denied Registation' in rhe current production firmware.

We have applied a workaround to the 'edge' firmware 3.3.002-46-g89963cee (or later). I suggest that anyone with this problem try to OTA update to that edge version, and see if it resolves/worksaround the issue for you.

My issue is slowly working itself loose. I've also learned to debug quite a bit on my own, as well as help on a ticket we're still working-through.

You can send "cellular cmd AT+CNMP=38" to restrict the modem to only LTE (the only technology still active in the US -- 2G and 3G have both been sunset, and night-light towers are possibly causing issues), and that may be part of straightening mine out. You can combine that with "log level debug cellular" (as above) to enable command outputs being displayed, and verify it's set with a "AT+CNMP?" command (note: not =?, but simply ?). It should return "+CNMP: 38" to indicate it's only allowed to play with LTE tech now. The modem stores this in NV memory, so it should persist through reboots and power cycles.

I've had spotty connectivity (95% off, 5% on) since the big event of whatever-changed on the network side that kicked me off... but I'm now seeing if restricting LTE-only has solved my issues. Fingers crossed!

edit: lol nope -- in the span of writing this, I've gone from "RegisteredRoaming" to "DeniedRegistration" again. The modem is absolutely reporting that the LTE service (T-Mobile Hologram -- the only service it now shows in a scan) is denying it. Back to Hologram support, then!