Mate 3 Cannot Recover from Ethernet Outage

Mate3 and Mate3s communications devices for Outback Power
User avatar
intellact
Forum Czar
Posts: 703
Joined: Mon Nov 28, 2005 3:20 am
Location: Ontario, Canada
Contact:

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by intellact » Mon May 08, 2017 4:56 am

larrywa wrote:You must be using pull technology to interrogate the Mate3 and herefore a different socket/connection/protocol.
It all depends on what the user selects:

When WattPlot is using the Network Data Stream (UDP/IP), the MATE3 pushes the data out to an IP port at the PC's IP address (hence no support for MATE3 commands).

When WattPlot is using the optional USB card, the MATE3 pushes the data out the USB cable, but that is a bidirectional serial data stream, so WattPlot can send commands to the MATE3. (The OutBack USB driver is not available for later versions of Windows, {but is not needed for Windows 10 - edit March 2018}.)

When WattPlot is using Modbus, it uses the MATE3's IP address to go directly into the MATE3 on a specified Modbus port and pulls out whatever data it wants, or changes whatever settings you choose.

I don't know what technology OpticsRE uses - still avoiding the rabbit hole, as I said :wink: - but I do know that WattPlot can access a significant number of OpticsRE performance and diagnostic values through the Modbus interface.

I hope that helps! :smile:
Last edited by intellact on Sun Mar 11, 2018 8:19 am, edited 1 time in total.
Andrew Welch, creator of WattPlot

outrigger999
Forum Guru
Posts: 105
Joined: Tue Oct 28, 2014 12:58 pm

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by outrigger999 » Mon May 08, 2017 11:20 am

outrigger999 wrote:How about this...Does anyone have a Mate 3 which stays alive more than a week? Longer? If so, how do you have the Mate 3 configured and what type of internet connection do you have? Which firmware version, settings, ethernet (wired, wifi extender, etc). I have a grid tie enphase system as well as off grid and the enlighten controller has been running more than 3 months straight on my internet connection without a hiccup so I know I have a solid internet connection. Today both are wired ethernet so i'm puzzled why I cannot get the mate 3 to maintain connectivity. Last night I lost connectivity in the middle of the night again. I'm thinking it could be the reboot feature so I've turned that off for now but if there's memory leaks in the software, then the mate 3 will go down no matter what.

I'm running one more test...I've installed a 10/100 ethernet hub near my Mate 3. I have a fairly long run of ethernet, no where near 328ft of course but I was thinking there could be two other issues. First, it could be signal strength so I'm putting a hub in the middle to repeat the signal and boost the signal strength. Second, I was thinking that gigabit ethenet could also be the culprit. Therefore I've purposely put a 10/100 ethernet hub in between to see if that solves my longevity issue by reducing the "speed" of my connection and going to a more standard ethernet signal back when the mate 3 was built. Right now I have a good signal, OpticsRE shows my system to be online, and I'll see how long I last. I'll report back what I find out.
Ok, so one day later after I posted this, I'm still up and running without a hiccup. Since I changed a few things (and not one like I should have) it's hard to say what's made the difference but in my opinion it was both repeating the singnal (increasing signal strength) and slowing down from gigabit to 10/100. The mate seems to be operating at 100Mbit. I have had to reset my Mate 3 almost every day. Occasionally, I would go more than 1 day but for the most part I would need to baby sit this device. If I can go for multiple days and even a week, that would great. I still have not heard if people out there are going for a week or longer without any modifications. If there is someone out there, please tell me how you're doing it! Thanks...

outrigger999
Forum Guru
Posts: 105
Joined: Tue Oct 28, 2014 12:58 pm

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by outrigger999 » Tue May 09, 2017 11:06 am

outrigger999 wrote:
outrigger999 wrote:How about this...Does anyone have a Mate 3 which stays alive more than a week? Longer? If so, how do you have the Mate 3 configured and what type of internet connection do you have? Which firmware version, settings, ethernet (wired, wifi extender, etc). I have a grid tie enphase system as well as off grid and the enlighten controller has been running more than 3 months straight on my internet connection without a hiccup so I know I have a solid internet connection. Today both are wired ethernet so i'm puzzled why I cannot get the mate 3 to maintain connectivity. Last night I lost connectivity in the middle of the night again. I'm thinking it could be the reboot feature so I've turned that off for now but if there's memory leaks in the software, then the mate 3 will go down no matter what.

I'm running one more test...I've installed a 10/100 ethernet hub near my Mate 3. I have a fairly long run of ethernet, no where near 328ft of course but I was thinking there could be two other issues. First, it could be signal strength so I'm putting a hub in the middle to repeat the signal and boost the signal strength. Second, I was thinking that gigabit ethenet could also be the culprit. Therefore I've purposely put a 10/100 ethernet hub in between to see if that solves my longevity issue by reducing the "speed" of my connection and going to a more standard ethernet signal back when the mate 3 was built. Right now I have a good signal, OpticsRE shows my system to be online, and I'll see how long I last. I'll report back what I find out.
Ok, so one day later after I posted this, I'm still up and running without a hiccup. Since I changed a few things (and not one like I should have) it's hard to say what's made the difference but in my opinion it was both repeating the singnal (increasing signal strength) and slowing down from gigabit to 10/100. The mate seems to be operating at 100Mbit. I have had to reset my Mate 3 almost every day. Occasionally, I would go more than 1 day but for the most part I would need to baby sit this device. If I can go for multiple days and even a week, that would great. I still have not heard if people out there are going for a week or longer without any modifications. If there is someone out there, please tell me how you're doing it! Thanks...
Ok, one more day and I'm still up! I'm sure it's not a record however more often than not I've needed to reset my Mate 3 already by this point so something I did was right. I'd be interested to hear from Outback to see if they know which of my two changes made the difference...

1. Added a repeater to boost signal strength
2. Moved to a 10/100 signal instead of gigabit. Obviously my gigabit switch supports 100Mbit and downgrades that port however perhaps using a hub which is actually only 10/100 might have made a difference.

I'm using a Netgear 5 port 10/100 hub.

Anyway, day 3 up and running.

User avatar
David LeBow
OutBack Management
Posts: 548
Joined: Sun May 03, 2015 10:42 am
Location: Bellingham, WA

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by David LeBow » Tue May 09, 2017 11:17 am

outrigger999 wrote: I'd be interested to hear from Outback to see if they know which of my two changes made the difference...

1. Added a repeater to boost signal strength
2. Moved to a 10/100 signal instead of gigabit. Obviously my gigabit switch supports 100Mbit and downgrades that port however perhaps using a hub which is actually only 10/100 might have made a difference.
The answer should be: neither. The most likely answer is: 1). If the signal strength was weak enough to drop periodically, it would be like poking the Mate3 in the eye with a stick, aggravating the issues it has in some environments with reconnecting. The switch speed issue shouldn't be the problem, but... any port in a storm. I'm interested in an update as you have it.

In my (non-Mate3) environment at home, I have a repeater installed, but that can sometimes drop the signal. Don't remember the brand. Keep us posted.
David LeBow
Sr. Director, Software Engineering


Alpha Technologies Inc. / OutBack Power Technologies
3767 Alpha Way Bellingham WA 98226 USA
http://www.alpha.com

JRHill
Forum Czar
Posts: 859
Joined: Thu Feb 17, 2011 11:12 am

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by JRHill » Tue May 09, 2017 11:39 am

David LeBow wrote:In my (non-Mate3) environment at home...
David, thanks for all you do, really. I'd be glad to help, too, as I want to move to the AXS and need a new home for my Mate3. I'll make you a great deal. ;-)

Best,
Jim

User avatar
David LeBow
OutBack Management
Posts: 548
Joined: Sun May 03, 2015 10:42 am
Location: Bellingham, WA

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by David LeBow » Tue May 09, 2017 11:58 am

JRHill wrote:I want to move to the AXS and need a new home for my Mate3. I'll make you a great deal. ;-)
You'd also need to make me a great deal on some solar panels, an inverter and, probably most important, some sunshine. A) I'm in the Pacific Northwest; B) my house is on the north side of a sloped plot of trees. Ixnay on the Unsay.

Alternatively, I suppose you could just make me a good deal on a chainsaw...
David LeBow
Sr. Director, Software Engineering


Alpha Technologies Inc. / OutBack Power Technologies
3767 Alpha Way Bellingham WA 98226 USA
http://www.alpha.com

outrigger999
Forum Guru
Posts: 105
Joined: Tue Oct 28, 2014 12:58 pm

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by outrigger999 » Wed May 10, 2017 11:07 am

Ok, one more day where nothing has disconnected and I have all my data! I think I've solved my problem.

User avatar
David LeBow
OutBack Management
Posts: 548
Joined: Sun May 03, 2015 10:42 am
Location: Bellingham, WA

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by David LeBow » Wed May 10, 2017 11:11 am

Many thanks for the feedback, Scott!

D.
David LeBow
Sr. Director, Software Engineering


Alpha Technologies Inc. / OutBack Power Technologies
3767 Alpha Way Bellingham WA 98226 USA
http://www.alpha.com

outrigger999
Forum Guru
Posts: 105
Joined: Tue Oct 28, 2014 12:58 pm

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by outrigger999 » Wed May 10, 2017 1:05 pm

David, I do feel I need to point out that I downgraded my Mate 3 firmware to 003.011.000. I pointed this out earlier but this is relevant to someone who's having the same issues I'm having.

User avatar
David LeBow
OutBack Management
Posts: 548
Joined: Sun May 03, 2015 10:42 am
Location: Bellingham, WA

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by David LeBow » Wed May 10, 2017 1:08 pm

So we've changed three different things. Do you feel that - after changing the Internet environment - an attempt at 3.15.3 would be worthwhile again? It would be great to isolate the issue a little further.

By the way - the Mate3 team is testing a 3.15.5 with a planned release in the next two weeks.
David LeBow
Sr. Director, Software Engineering


Alpha Technologies Inc. / OutBack Power Technologies
3767 Alpha Way Bellingham WA 98226 USA
http://www.alpha.com

outrigger999
Forum Guru
Posts: 105
Joined: Tue Oct 28, 2014 12:58 pm

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by outrigger999 » Wed May 10, 2017 6:14 pm

David LeBow wrote:So we've changed three different things. Do you feel that - after changing the Internet environment - an attempt at 3.15.3 would be worthwhile again? It would be great to isolate the issue a little further.

By the way - the Mate3 team is testing a 3.15.5 with a planned release in the next two weeks.
I can certainly try 3.15.3 however I don't think that's posted anymore as someone pointed out. I think what I'll do is wait for this to go a week, which means this coming Monday and if we go that long then I'll adjust the firmware. If you want me to test 3.15.3, then please send that to me or repost it.

petertearai
Forum Guru
Posts: 397
Joined: Fri Aug 06, 2010 4:35 pm
My RE system: 2220 watt pv, fm80, outback fxr 2000 watt inverter. Shoto 12 times 2 volt exc400 lead carbon batterys 24 volt system victron battery moniter. optics RE
Location: NewZealand North Island

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by petertearai » Thu May 11, 2017 12:51 am

connected optics a week or so ago . running on a huawe 4g wireless internet box with cat 6 cable to mate 3 . Hasn't missed a beat as far as i can see. cable about 8 meters.
2220 pv. fm80. 12 times 2 volt exc shoto lead carbon 450 AH 24 volts. victron battery monitor. outback fxr 2000 watt inverter

User avatar
David LeBow
OutBack Management
Posts: 548
Joined: Sun May 03, 2015 10:42 am
Location: Bellingham, WA

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by David LeBow » Thu May 11, 2017 8:45 am

outrigger999 wrote:I can certainly try 3.15.3 however I don't think that's posted anymore as someone pointed out.
And of course... with "3.15.3" I mean "3.15.1".

D.
David LeBow
Sr. Director, Software Engineering


Alpha Technologies Inc. / OutBack Power Technologies
3767 Alpha Way Bellingham WA 98226 USA
http://www.alpha.com

outrigger999
Forum Guru
Posts: 105
Joined: Tue Oct 28, 2014 12:58 pm

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by outrigger999 » Thu May 11, 2017 10:54 am

Completely stable now, again overnight no issue and fully online this morning.

outrigger999
Forum Guru
Posts: 105
Joined: Tue Oct 28, 2014 12:58 pm

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by outrigger999 » Sun May 14, 2017 3:59 pm

Well, I'll consider this the best run I've had so far but on Saturday night at around 9pm the system went offline. Still, it was the best run I've had to date. As I bring the system back online right now, I'll put the current firmware on the device and run the test again.

outrigger999
Forum Guru
Posts: 105
Joined: Tue Oct 28, 2014 12:58 pm

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by outrigger999 » Sun May 14, 2017 4:25 pm

Ok, as of 1:15pm HST (Hawaii Standard Time or UTC -10) on 5/14/17 I reset the Mate 3 and installed the latest available firmware which is 003.015.001. Let's see how long this one goes for. Incidentally, I'd like to say that when installing the new firmware, I didn't reboot the Mate 3 first and the firmware update got stuck at initializing the ethernet processor (very first step). This means that the hardware, at least the ethernet chipset was hung. While the Mate seemed to be performing it's other tasks, I needed to reboot it first before updating the firmware.

I should also state that my VFX3648s (all 4 of them) are not recognized and I need to reset them in the settings for OpticsRE before I get a functioning dashboard. Further, all my previous data (I assume) which has been "collected" by the system during the Mate 3 outage is compiled together in one large green bar on the chart. I've seen all this behavior before so this isn't exactly new but since I was doing a reset where I was reporting what I was doing, I thought I'd mention all those items again.

bluespur
Forum Expert
Posts: 68
Joined: Wed Feb 23, 2011 2:32 pm
My RE system: VFX3024x2
Flexmax60x2
3000W total solar, 135w panel array and 220w panel array
Mate3
12xHawker Perfect Plus 8PZS120 FLA batteries
Own built generator Kubota D600 engine, 7.5Kw gen head

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by bluespur » Sun May 14, 2017 8:02 pm

Hi David,
Please tell us if 3.15.5 is expected to improve the Mate3 connectivity issue. (and is there somewhere I can register to be notified?)
And I now have a different manifestation of the problem, my device is offline according to OpticsRE, but the data is still being updated. Never happened before like this up up until last week when I let it run for a couple of days before manually rebooting the mate, and now its doing it again. At least my data is getting there :)
Cheers
Garth

User avatar
David LeBow
OutBack Management
Posts: 548
Joined: Sun May 03, 2015 10:42 am
Location: Bellingham, WA

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by David LeBow » Mon May 15, 2017 10:26 am

Hi Garth,

"Please tell us if 3.15.5 is expected to improve the Mate3 connectivity issue. "

It is expected to improve connectivity and we've made some changes to make the Auto-Reboot interval processing more robust. I wouldn't be surprised if there are still some configurations which cause us connectivity problems. For those that aren't solved with 3.15.5, I'd be interested in a couple of network-savvy users who would like to help us "wireshark" their setup.

"(and is there somewhere I can register to be notified?)"

Not exactly. I'll make sure to post a notification to the forum and to the log-on message on Optics.

"my device is offline according to OpticsRE, but the data is still being updated"

I've fired off an e-mail to have your system looked at a bit more closely to help determine what's going on. The Silver Lining: the data's coming through, I suppose...

David
David LeBow
Sr. Director, Software Engineering


Alpha Technologies Inc. / OutBack Power Technologies
3767 Alpha Way Bellingham WA 98226 USA
http://www.alpha.com

User avatar
David LeBow
OutBack Management
Posts: 548
Joined: Sun May 03, 2015 10:42 am
Location: Bellingham, WA

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by David LeBow » Mon May 15, 2017 10:44 am

"And I now have a different manifestation of the problem, my device is offline according to OpticsRE, but the data is still being updated."

This just in from the team:

The Mate isnÔÇÖt sending ping commands but continues to send status data. This is a known bug with 3.15.3. Fail safes were put into the 3.15.5 firmware, which appear to fix this problem.


D.
David LeBow
Sr. Director, Software Engineering


Alpha Technologies Inc. / OutBack Power Technologies
3767 Alpha Way Bellingham WA 98226 USA
http://www.alpha.com

outrigger999
Forum Guru
Posts: 105
Joined: Tue Oct 28, 2014 12:58 pm

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by outrigger999 » Mon May 15, 2017 11:21 am

This morning my Mate 3 is still online and as I mentioned earlier running the latest "available" firmware however I'm missing bars in my chart. I'm thinking of restoring the previous firmware since this latest firmware seems to have other issues.

bluespur
Forum Expert
Posts: 68
Joined: Wed Feb 23, 2011 2:32 pm
My RE system: VFX3024x2
Flexmax60x2
3000W total solar, 135w panel array and 220w panel array
Mate3
12xHawker Perfect Plus 8PZS120 FLA batteries
Own built generator Kubota D600 engine, 7.5Kw gen head

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by bluespur » Mon May 15, 2017 2:16 pm

Hi David,
Thanks for the update, appreciated, odd that this has chosen to manifest itself now after running 3.15.3 since its release.
I may be able to assist with wiresharking, but would I be correct in thinking that to do so I would need to have a PC in close proximity to the MAte3?
Cheers
Garth

User avatar
David LeBow
OutBack Management
Posts: 548
Joined: Sun May 03, 2015 10:42 am
Location: Bellingham, WA

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by David LeBow » Mon May 15, 2017 2:48 pm

"but would I be correct in thinking that to do so I would need to have a PC in close proximity to the MAte3?"

Yeah... that'd be pretty useful.
David LeBow
Sr. Director, Software Engineering


Alpha Technologies Inc. / OutBack Power Technologies
3767 Alpha Way Bellingham WA 98226 USA
http://www.alpha.com

mtnmerlin
Forum Member
Posts: 12
Joined: Tue Jan 26, 2016 1:11 pm
My RE system: Mate3
CC Flexmax 80
INV Flexware 500
eight 170RE AGM batteries
six Suniva OPT 265 W modules

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by mtnmerlin » Wed May 31, 2017 1:33 pm

I have updated to 3.15.5. I have the reboot period set to 4 hours. I will see how well it works.

User avatar
SEYLIONE
Forum Guru
Posts: 125
Joined: Thu Mar 22, 2012 9:16 am
My RE system: Off grid house in the southern french Alps 1740m above sea
level. Commissioned 1st Jul 2012.

4 Bosch 245Wp Panels, 12 OPzS 2V 1 220Ah batteries
Management by OutbackPower: FP1-34 including:
FlexMax 80 Charger, GVFX3024E Converter-charger,
FlexNet DC, Hub, Mate3.

Monitoring every 3 hours over Internet (lsdx.eu/GPV_ZSDX)
by 3G managed by a local microcomputer and router.
Location: Serre-Chevalier, France

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by SEYLIONE » Wed Jun 07, 2017 11:10 pm

Hello everybody,

I have had 3.15.5 set at 24H reboot for the last 3 days. Does not hold: nothing on my computer. Last transmission 4 hours ago though it should be every Hour).
Any one has tried the new Mate3S?

Thanks for info
Lionel

sodamo
Forum Czar
Posts: 779
Joined: Mon Apr 03, 2006 1:55 am
My RE system: Totally off grid - Hamakua side, Big Island, Hi

36 Trina 280 & 16 Phono 250 in 48 volt array (fixed) 14080w
Radians GG8048/GS4048 w GSLCs
4 FM 80 charge controllers
Hub 10.3
FNDC w/3 shunts
Mate 3s
OpticsRE - MMKL - Ninole Hi
12 SimpliPhi 3.8
14Kw Kohler 14RESA Propane

Charging Only
2FX3048T
Hub
FNDC. - shared shunt feeds from Radian FNDC
Mate 3s
10Kw MEP 803a Military gen

Honda EU7000i

Davis VantagePro2 Wx Station
On line at: https://www.wunderground.com/dashboard/pws/KHININOL2
Location: Ninole, Hi

Re: Mate 3 Cannot Recover from Ethernet Outage

Post by sodamo » Fri Jun 09, 2017 10:26 pm

Would love to read the the new Mate 3S is indeed a fix.
David
Please visit http://vacation.ninolehawaii.com

Post Reply