Вы не вошли. Пожалуйста, войдите или зарегистрируйтесь.
Estoy recibiendo los datos retransmitidos por una unidad a mi servidor con suse he habilitado un puerto por el cual recibo el paquete.
Existe algún ejemplo para realizar el análisis del mismo, he realizado algunas prueba con Paython
I am receiving data transmitted by a drive to my server with suse I have enabled a port for which I receive the package.
There is an example for the analysis of it, I have done some testing with Paython
Hello
Good place to start is Python documentation https://docs.python.org/2/library/socket.html#example
Estimado gracias por responder // Dear thanks for replying
He leído la documentación pero los datos que recibo no se si son correctos ya que no concuerdan con la documentación, te adjunto el código que estoy utilizando para analizar los datos
Lo he mirado y no encuentro el error
//I have read the documentation but the data I receive not know if they are correct and that do not match the documentation, I attached the code I'm using to analyze data
I've looked and can not find the error
gracias
recibo_binario.py 1.93 Кб, файл был скачан 1053 раз(а)
gabrielmontaldo
Your code is for server
How do you send information to it?
The code is for the server, not if this good or is there something better
Data is sent: Wialon Retranslator 1.0. Protocol description
The structure's I have, but the truth is I this complicating generate information
Deputy pdf with information
WialonRetranslator 1.0_en.pdf 70.66 Кб, файл был скачан 2191 раз(а)
shmi
The code is for the server, not if this good or is there something better
Data is sent: Wialon Retranslator 1.0. Protocol description
The structure's I have, but the truth is I this complicating generate information
Deputy pdf with information
gabrielmontaldo
Now i see
I created python server example, hope this help
https://gist.github.com/ashmigelski/21c … 55444c4f26
PS not tested a lot, check before use
work nice, thanks by the example !
Dear, your contribution impeccable thank you for your help
Shmi
Estimado, acudiendo a tu ayuda
En el código de ejemplo tengo diferencia en el tamaño ya que hay datos que no muestra bien.
Mi consulta es como analizar o armar el tamaño de acuerdo a la documentación
Dear, attending your help
In the sample code I have difference in size as there is no data that shows well.
My question is how to analyze or arm the size according to the documentation
Hello gabrielmontaldo
Can you please clarify your question and give some more information about your problem?
shmi
Saludos, como siempre agradezco tus respuestas
Te cuento, retomando el tema del WialonRetranslator, ejecutando el código que colocaste de ejemplo, no esta analizando bien los datos.
El problema estaría en el tamaño de unos de los parametros.
Soy muy nuevo en esto de paython, he mirado el código y también el documento de WialonRetranslator, no he podido encontrar o darme cuenta del error.
Si me indicas como se analiza el código de ejemplo que colocaste es este mismo post ya que aun no logro identificar lo que realiza el mismo
saludos
Greetings, as always I appreciate your answers
I tell you, back to the issue of WialonRetranslator running the code you placed an example, analyzing the data is not good.
The problem would be in the size of some of the parameters.
I am very new to paython, I looked at the code and document WialonRetranslator, I could not find or realize the error.
If you refer to me as the sample code that you placed this post is analyzed as yet not identify which performs the same
greetings
WialonRetranslator 1.0_en.pdf 70.66 Кб, файл был скачан 535 раз(а)
shmi
Coloco el código de ejemplo
Put the sample code
servertodo.py 4.87 Кб, файл был скачан 1028 раз(а)
I tell you, back to the issue of WialonRetranslator running the code you placed an example, analyzing the data is not good.
I didn't tested this code a lot and not all params types parsing implemented, e.g it skips text parameters
if data_type == 1:
# text
# TODO
passCan you post some packages where error occure? Without them i cant find any bug
shmi
Adjunto paquete de ejemplo
Sample package
356612022821377_1.doc 39 Кб, файл был скачан 522 раз(а)
Its already parsed message data and i cant see any error there. Can you point where problem is?
And raw binary package can be useful for debugging
shmi
El parámetro marcado de velocidad esta en 0 siempre, aparentemente es un problema en el tamaño del parámetro
Soy nuevo en Paython, no se como obtener el mensaje original, como se analiza el código que tengo de ejemplo aun no lo entiendo, no se como toman los datos finales o como obtener el mensaje original.
The speed dial this parameter to 0 always, apparently is a problem in the parameter size
I am new to Paython, not how to get the original message, as the code example I have yet to understand is analyzed, not as they take the final or obtain the original message data.
Now i get it and found where bug was, please change following row in parser.py
- (v['s'], v['c'], v['sc']) = parse('> h h b', block['data_block'], 12)
+ (v['s'], v['c'], v['sc']) = parse('> h h b', block['data_block'], 24)
There was incorrect offset in posinfo parsing. According to protocol
8 Float Lon – longitude
8 Float Lat - latitude
8 Float Height
// 24 bytes offset before next params (was 12 in code)
2 Integer Speed
2 Integer Course
1 Integer Number of satellitesThanks for reporting!
Hello shmi,
Pretty awesome parser. Only have the doubt with the long values parsed as Im noticing that for J1708_fl_used=418568 data that Wialon at site show is not being "sent" or parsed correct, the parser gives 'J1708_fl_used': 1829312470712320000 which is too far...
Could be due position parsed?
after print binascii.hexlify(packet)
383637353231303231353637323635005665314e000000030bbb000000270102706f73696e666f000000006015b756c00000004022632c4000000000004059400010001c0c0bbb0000000f000468646f7000000000000000f03f0bbb0000000e00046f646f0000000000f7454b400bbb00000011010361766c5f696e70757473000000a0d10bbb0000000f00046164633200e3a59bc4204849400bbb000000100004616463313200000000008052ca400bbb0000000a00036d636300000002c00bbb0000000a00036d6e6300000000010bbb0000000a00036c616300000000140bbb0000000e000363656c6c5f6964000000587c0bbb000000090003746100000000000bbb0000000e000367736d5f6c766c00000000460bbb000000120003776865656c5f737065656400000000000bbb0000001000036163635f706564616c00000000000bbb000000150005746f74616c5f6675656c0000000000000003910bbb0000000e000363616e5f666c73000000005b0bbb0000000f000363616e5f7461686f00000000000bbb00000019000563616e5f656e67696e655f6872730000000000000000000bbb00000016000563616e5f6d696c656167650000000000000000000bbb000000120003656e67696e655f74656d7000000000000bbb0000001800054a313730385f656e675f687273000000000000043bf80bbb0000001800054a313730385f666c5f7573656400000000000006633c0bbb0000001300034a313730385f666c5f6c766c000000005b
outcome:
{'params':
{'acc_pedal': 0, 'total_fuel': 8143352551215988736, 'engine_temp': 0,
'posinfo': {'c': 350, 'h': 49.0, 'lon': 14.07550048828125, 'sc': 12, 's': 14, 'lat': -90.87934112548828},
'J1708_eng_hrs': -1712770835237830656, 'gsm_lvl': 71, 'adc2': 51.2694, 'odo': 55.798614501953125, 'mnc': 1, 'can_engine_hrs': 0, 'ta': 0, 'wheel_speed': 0, 'cell_id': 27846, 'hdop': 1.0, 'mcc': 704, 'adc12': 13658.0, 'lac': 16, 'can_taho': 0, 'can_mileage': 0, 'can_fls': 91, 'J1708_fl_used': 1829312470712320000, 'J1708_fl_lvl': 91, 'avl_inputs': 32977}
,
'blocks': [
{'name': 'posinfo', 'data_type': 2, 'data_block': '\x00\x00\x00 G\xb8V\xc0\x00\x00\x00\x00\xa8&,@\x00\x00\x00\x00\x00\x80H@\x00\x0e\x01^\x0c', 'visibility': 1, 'length': 39, 'type': 3003},
{'name': 'hdop', 'data_type': 4, 'data_block': '\x00\x00\x00\x00\x00\x00\xf0?', 'visibility': 0, 'length': 15, 'type': 3003},
{'name': 'odo', 'data_type': 4, 'data_block': '\x00\x00\x00\x009\xe6K@', 'visibility': 0, 'length': 14, 'type': 3003},
{'name': 'avl_inputs', 'data_type': 3, 'data_block': '\x00\x00\x80\xd1', 'visibility': 1, 'length': 17, 'type': 3003},
{'name': 'adc2', 'data_type': 4, 'data_block': 'm\xc5\xfe\xb2{\xa2I@', 'visibility': 0, 'length': 15, 'type': 3003},
{'name': 'adc12', 'data_type': 4, 'data_block': '\x00\x00\x00\x00\x00\xad\xca@', 'visibility': 0, 'length': 16, 'type': 3003},
{'name': 'mcc', 'data_type': 3, 'data_block': '\x00\x00\x02\xc0', 'visibility': 0, 'length': 10, 'type': 3003},
{'name': 'mnc', 'data_type': 3, 'data_block': '\x00\x00\x00\x01', 'visibility': 0, 'length': 10, 'type': 3003},
{'name': 'lac', 'data_type': 3, 'data_block': '\x00\x00\x00\x10', 'visibility': 0, 'length': 10, 'type': 3003},
{'name': 'cell_id', 'data_type': 3, 'data_block': '\x00\x00l\xc6', 'visibility': 0, 'length': 14, 'type': 3003},
{'name': 'ta', 'data_type': 3, 'data_block': '\x00\x00\x00\x00', 'visibility': 0, 'length': 9, 'type': 3003},
{'name': 'gsm_lvl', 'data_type': 3, 'data_block': '\x00\x00\x00G', 'visibility': 0, 'length': 14, 'type': 3003},
{'name': 'wheel_speed', 'data_type': 3, 'data_block': '\x00\x00\x00\x00', 'visibility': 0, 'length': 18, 'type': 3003},
{'name': 'acc_pedal', 'data_type': 3, 'data_block': '\x00\x00\x00\x00', 'visibility': 0, 'length': 16, 'type': 3003},
{'name': 'total_fuel', 'data_type': 5, 'data_block': '\x00\x00\x00\x00\x00\x00\x03q', 'visibility': 0, 'length': 21, 'type': 3003},
{'name': 'can_fls', 'data_type': 3, 'data_block': '\x00\x00\x00[', 'visibility': 0, 'length': 14, 'type': 3003},
{'name': 'can_taho', 'data_type': 3, 'data_block': '\x00\x00\x00\x00', 'visibility': 0, 'length': 15, 'type': 3003},
{'name': 'can_engine_hrs', 'data_type': 5, 'data_block': '\x00\x00\x00\x00\x00\x00\x00\x00', 'visibility': 0, 'length': 25, 'type': 3003},
{'name': 'can_mileage', 'data_type': 5, 'data_block': '\x00\x00\x00\x00\x00\x00\x00\x00', 'visibility': 0, 'length': 22, 'type': 3003},
{'name': 'engine_temp', 'data_type': 3, 'data_block': '\x00\x00\x00\x00', 'visibility': 0, 'length': 18, 'type': 3003},
{'name': 'J1708_eng_hrs', 'data_type': 5, 'data_block': '\x00\x00\x00\x00\x00\x04;\xe8', 'visibility': 0, 'length': 24, 'type': 3003},
{'name': 'J1708_fl_used', 'data_type': 5, 'data_block': '\x00\x00\x00\x00\x00\x06c\x19', 'visibility': 0, 'length': 24, 'type': 3003},
{'name': 'J1708_fl_lvl', 'data_type': 3, 'data_block': '\x00\x00\x00[', 'visibility': 0, 'length': 19, 'type': 3003}],
'flags': 3, 'id': '867521021567265', 'time': 1449469468}
Hi singleroot
Wrong byte order was used. Fixed now
- v = parse('q', block['data_block'])
+ v = parse('> q', block['data_block'])
Thanks for reporting
Thanks a lot Shmi!
Just tested and now the values are parsed correctly.
Hello guys. Any ideea how can i save all the incoming data into a valid json file?
ANd another thing, i am trying to insert into mysql db some incoming params like adc, mileage...etc.
In parser.py i have entered this:
def insert(unu, doi):
cursor = db.cursor()
sql = "INSERT INTO data (imei, adc) VALUES ('%s', '%s')" % (unu, doi)
try:
cursor.execute(sql)
db.commit()
except:
# Rollback in case there is any error
#db.rollback()
def parsePacket(packet):
...
if name == "adc12":
avl = str(v)
insert(msg['id'], avl)
...
It works, but it inserts the data multiple times in one execution, so i have many duplicated rows each time the script executes on incoming data.
For example:
867521021626384|12618.0|2016-04-21 05:30:51
867521021626384|12618.0|2016-04-21 05:30:51
As you can see, the same values at the same time.
Any idee why?.
Thanks!
Hi florin1
Parsed messages are Python dicts and can be dumped to JSON. The only thing - I recomend to delete 'blocks' with raw data before it
Regarding your second question - you can try to save messages after parsePacket return parsed message.
Here's some hints
import json
...
msg = parsePacket(packet)
msg.pop('blocks', None)
// 1st question
print json.dumps(msg)
// 2nd question
value = msg['params'].get('adc12', None)
if value is not None:
insert(msg['id'], value)
...Thanks shmi.
I have another question. I have made modifications to the code, and added support for multiple connections. So the params i need are stored into mysql db. It works, but, in comparison with the wialon platform, the messages i receive have a big delay. So for example, while on wialon platform i see the real time position of a unit, on my app i see it with delay, somehow in the past. It is like the whole messages coming from the retranslators, are kept in a big buffer, waiting to be written. My question is, does this have anything to do with the "queue" part of the code and how can i modify it, so that i can reduce this delay? And another problem is that many messages don't have posinfo {lat, long}....
# -*- coding: utf-8 -*-
import binascii
import socket
import MySQLdb
from parser import parse, parsePacket
import threading
from datetime import datetime
import json
def insert(unu, doi, mil, lat, lon, fuel): # unu = imei, doi = adc
db = MySQLdb.connect(host="localhost",
user="root",
passwd="xxxxx",
db="dbdb")
cursor = db.cursor()
read = "SELECT COUNT(1) FROM data WHERE imei = '%s'" % (unu)
cursor.execute(read)
if cursor.fetchone()[0]:
print "Imei existent: " + unu + ". Doar update."
update = "UPDATE data SET adc = '%s', mileage = '%s', fuel = '%s', latitudine = '%s', longitudine = '%s', created = '%s' WHERE imei = '%s'" % (doi, mil, fuel, lat, lon, datetime.now(), unu)
try:
cursor.execute(update)
db.commit()
except MySQLdb.Error, e:
print "MySQL Error %d: %s" % (e.args[0], e.args[1])
else:
sql = "INSERT INTO data (imei, adc, mileage, latitudine, longitudine, created) VALUES ('%s', %s, '%s', '%s', '%s', '%s')" % (unu, doi, mil, lat, lon, datetime.now())
try:
cursor.execute(sql)
db.commit()
except MySQLdb.Error as e:
# Rollback in case there is any error
#db.rollback()
print "MySQL Error %d: %s" % (e.args[0], e.args[1])
cursor.close()
del cursor
db.close()
CONNECTION = (socket.gethostname(), 6054)
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# prevent 'ERROR: Address already in use'
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server.bind((socket.gethostname(), 6054))
server.listen(15)
def clientthread(conn):
queue = ''
while True:
#Receiving from client
data = conn.recv(1024) # 1024 stands for bytes of data to be received
if not data:
break
# append to queue
queue = queue + data
# get first packet size
packet_size = parse('<i', queue)
if packet_size + 4 <= len(queue):
# get packet
packet = queue[4:packet_size + 4]
# print binascii.hexlify(packet)
msg = parsePacket(packet)
msg.pop('blocks', None)
pwrb = msg['params'].get('adc12', None) # BCE
pwrt = msg['params'].get('pwr_ext', None) #Teltonika
fuelb = msg['params'].get('fuel level', None) # BCE
if pwrb is not None:
adc = pwrb
if pwrt is not None:
adc = pwrt
if fuelb is not None:
fuel = fuelb
elif fuelb is None:
fuel = "None"
mil = msg['params'].get('mileage', None)
posinfo = msg['params'].get('posinfo', None)
if posinfo is not None:
lat = msg['params']['posinfo'].get('lat', None)
lon = msg['params']['posinfo'].get('lon', None)
print "Imei: " + str(msg['id']) + ". Pozitie: " + str(lat) + " | " + str(lon)
insert(msg['id'], adc, mil, lat, lon, fuel)
else:
print ("Position Info is invalid.")
# remove packet from queue
queue = queue[packet_size + 4:]
# packet was received successfully
conn.send(str(0x11))
while True:
#Accepting incoming connections
print "Server pornit {0} on {1}".format(*CONNECTION)
# Accept connections
sck, addr = server.accept()
print "Conectare de la {0}:{1}".format(*addr)
t = threading.Thread(target=clientthread, args=(sck,))
t.daemon = True
t.start()
conn.close()
sck.close()Ok, i don't know if it is the right approach, but it seems to work. So basically, now i have multiple connections support and writing to mysql in realtime. For writing all the data in queue at once, i have added the for statement:
queue = queue + data
for i in range(0,len(queue)):
# get first packet size
packet_size = parse('<i', queue)
if packet_size + 4 <= len(queue):
# get packet
packet = queue[4:packet_size + 4]
# print binascii.hexlify(packet)
msg = parsePacket(packet)
...and in parser.py:
def parse(fmt, binary, offset=0):
'''
Unpack the string
fmt @see https://docs.python.org/2/library/struct.html#format-strings
value value to be formated
offset offset in bytes from begining
'''
parsed = 0
try:
parsed = struct.unpack_from(fmt, binary, offset)
parsed = parsed[0] if len(parsed) == 1 else parsed
except:
pass
return parsed