Skip to content

Commit c090c58

Browse files
cat-hainescat-haines
committed
Merge pull request #1 from electricimp/v1.0.0
V1.0.0
2 parents b86f1fe + dde6a13 commit c090c58

File tree

2 files changed

+239
-68
lines changed

2 files changed

+239
-68
lines changed

‎README.md

+134
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
# Serializer 1.0.0
2+
3+
The Serializer call includes two `static` methods that allow you serialize (nearly) any Squirrel object into a blob, and deserialize perviously serialized objects. This is particulairly useful if you're planning to store information with [hardware.spiflash](https://electricimp.com/docs/api/hardware/spiflash) or the with the [SPIFlash Library](https://github.com/electricimp/spiflash/tree/v1.0.0).
4+
5+
*NOTE:* The *Serializer* class only uses `static` methods, and as a result does not to be initialized through a constructor.
6+
7+
**To add this library to your project, add `#require "Serializer.class.nut:1.0.0"`` to the top of your device code.**
8+
9+
You can view the library’s source code on [GitHub](https://github.com/electricimp/serializer/tree/v1.0.0).
10+
11+
## Serializable Squirrel
12+
13+
The Serializer class currently supports the following types:
14+
15+
- [arrays](https://electricimp.com/docs/squirrel/array/)
16+
- [blobs](https://electricimp.com/docs/squirrel/blob/)
17+
- [booleans](https://electricimp.com/docs/squirrel/bool/)
18+
- [floats](https://electricimp.com/docs/squirrel/float/)
19+
- [integers](https://electricimp.com/docs/squirrel/integer/)
20+
- [strings](https://electricimp.com/docs/squirrel/string/)
21+
- [tables](https://electricimp.com/docs/squirrel/table/)
22+
- `null`
23+
24+
The Serializer cannot serialize the following types:
25+
26+
- `function` - Functions / function pointers.
27+
- `instance` - Class instances.
28+
- `meta` - Meta objects such as *device* and *hardware*.
29+
30+
## Class Methods
31+
32+
### Serializer.serialize(*obj, [prefix]*)
33+
34+
The *Serializer.serialize* method allows you to transform an arbitrary Squirrel object (*obj*) into a blob.
35+
36+
```squirrel
37+
# require "Serializer.class.nut:1.0.0"
38+
39+
local data = {
40+
"foo": "bar",
41+
"timestamps": [ 1436983175, 1436984975, 1436986775, 1436988575, 1436990375],
42+
"readings": [ 32.5, 33.6, 32.8, 32.9, 32.5 ],
43+
"otherData": {
44+
"state": true,
45+
"test": "test"
46+
}
47+
}
48+
49+
local serializedData = Serializer.serialize(data);
50+
51+
// Write the data to SPI Flash @ 0x0000
52+
spiFlash.enable();
53+
spiFlash.erasesector(0x0000);
54+
spiFlash.write(0x0000, serializedData, SPIFLASH_PREVERIFY | SPIFLASH_POSTVERIFY);
55+
spiFlash.disable();
56+
```
57+
58+
If a *prefix* was passed to the method, the Serializer will write this data at the beginning of the blob. Immediatly proceeding the prefix data, the Serializer will write 3 bytes of header information: a 16-bit unsigned integer representing the length of the serialized data, and an 8-bit unsigned integer representing a CRC.
59+
60+
| Byte | Description |
61+
| ---- | ---------------------------- |
62+
| 0 | The lower byte of the length |
63+
| 1 | The upper byte of the length |
64+
| 2 | The CRC byte |
65+
66+
**NOTE:** The 16-but length value does include the length of the prefix (if included) or the header data (3 bytes).
67+
68+
### Serializer.deserialize(*serializedBlob, [prefix]*)
69+
70+
The *Serializer.deserialize* method will deserialize a blob that was previous serialized with the *Serializer.serialize* method. If the blob was serialized with a *prefix*, the same *prefix* must be passed into the *Serializer.deserialize* method.
71+
72+
```squirrel
73+
# require "Serializer.class.nut:1.0.0"
74+
75+
// Setup SpiFlash object
76+
// ...
77+
78+
spiFlash.enable();
79+
80+
// Read the header information
81+
local dataBlob = spiFlash.read(0x00, 3);
82+
// Get the length from the first two bytes
83+
local len = dataBlob.readn('w');
84+
85+
// Move to the end of the blob
86+
dataBlob.seek(0, 'e');
87+
88+
// Read the length of the data starting at the end of the header
89+
spiFlash.readintoblob(0x03, dataBlob, len);
90+
91+
// Disable the SPIFlash since we're done
92+
spiFlash.disable();
93+
94+
// Deserialize the blob
95+
local data = Serializer.deserialize(dataBlob);
96+
97+
98+
99+
// Log some data to make sure it worked:
100+
server.log(data.foo); // bar
101+
server.log(data.otherData.state); // true
102+
server.log(data.otherData.test); // test
103+
104+
server.log("Readings:");
105+
for(local i = 0; i < data.timestamps.len(); i++) {
106+
server.log(data.timestamps[i] + ": " + data.readings[i]);
107+
}
108+
109+
```
110+
111+
### Serializer.sizeof(obj)
112+
113+
The *Serializer* class needs to add a variety of metadata to Serialized objects in order to properly know how to deserialize the blobs. The *Serializer.sizeof* method can be used to quickly determin the size of an object after serialization.
114+
115+
```squirrel
116+
# require "Serializer.class.nut:1.0.0"
117+
118+
local data = {
119+
"foo": "bar",
120+
"timestamps": [ 1436983175, 1436984975, 1436986775, 1436988575, 1436990375],
121+
"readings": [ 32.5, 33.6, 32.8, 32.9, 32.5 ],
122+
"otherData": {
123+
"state": true,
124+
"test": "test"
125+
}
126+
}
127+
128+
// Check how large our blob will be before serializing
129+
server.log(Serializer.sizeof(data));
130+
```
131+
132+
# License
133+
134+
The Serializer class is licensed under [MIT License](https://github.com/electricimp/serializer/tree/master/LICENSE).

‎Serializer.class.nut

+105-68
Original file line numberDiff line numberDiff line change
@@ -1,43 +1,119 @@
1-
// =============================================================================
1+
// Copyright (c) 2015 Electric Imp
2+
// This file is licensed under the MIT License
3+
// http://opensource.org/licenses/MIT
4+
25
class Serializer {
3-
static version = [0,1,0];
6+
static version = [1,0,0];
47

58
// Serialize a variable of any type into a blob
6-
function serialize (obj, prefix = null) {
9+
static function serialize (obj, prefix = null) {
710
// Take a guess at the initial size
8-
local b = blob(2000);
11+
local b = blob(512);
912
local header_len = 3;
1013
local prefix_len = (prefix == null) ? 0 : prefix.len();
11-
// Write the prefix plus dummy data for len and crc late
14+
15+
// Write the prefix plus dummy data for length and CRC
1216
if (prefix_len > 0) {
1317
foreach (ch in prefix) b.writen(ch, 'b');
1418
}
15-
b.writen(0, 'b');
16-
b.writen(0, 'b');
17-
b.writen(0, 'b');
19+
20+
// Write two bytes for length
21+
b.writen(0, 'w'); // 0x0000
22+
// Write 1 byte for CRC
23+
b.writen(0, 'b'); // 0x00
24+
1825
// Serialise the object
1926
_serialize(b, obj);
20-
// Shrink it down to size
21-
b.resize(b.tell());
22-
// Go back and add the len and CRC
23-
local body_len = b.len() - header_len - prefix_len;
27+
28+
// Resize the blog as required
29+
local body_len = b.tell();
30+
b.resize(body_len);
31+
32+
// Remove header and prefix from length of body
33+
body_len -= (header_len + prefix_len);
34+
35+
// Write the body length and CRC
2436
b.seek(prefix_len);
2537
b.writen(body_len, 'w');
2638
b.writen(LRC8(b, header_len + prefix_len), 'b');
27-
// Hop back home
39+
40+
// Move pointer to start of blob,
41+
// and return serialized object
2842
b.seek(0);
2943
return b;
3044
}
3145

32-
function _serialize (b, obj) {
46+
// Deserialize a string into a variable
47+
static function deserialize (s, prefix = null) {
48+
// Read and check the prefix and header
49+
local prefix_len = (prefix == null) ? 0 : prefix.len();
50+
local header_len = 3;
51+
s.seek(0);
52+
53+
local pfx = prefix_len > 0 ? s.readblob(prefix_len) : null;
54+
local len = s.readn('w');
55+
local crc = s.readn('b');
56+
57+
if (s.len() != len+prefix_len+header_len) throw "Expected " + (len+prefix_len+header_len) + " bytes (got " + s.len() + " bytes)";
58+
// Check the prefix
59+
if (prefix != null && pfx.tostring() != prefix.tostring()) throw "Prefix mismatch";
60+
// Check the CRC
61+
local _crc = LRC8(s, prefix_len+header_len);
62+
if (crc != _crc) throw format("CRC err: 0x%02x != 0x%02x", crc, _crc);
63+
// Deserialise the rest
64+
return _deserialize(s, prefix_len+header_len).val;
65+
}
66+
67+
static function sizeof(obj, prefix = null) {
68+
local size = 3; // header
69+
if (prefix != null) size += prefix.len();
70+
return size += _sizeof(obj);
71+
}
3372

73+
static function _sizeof(obj) {
74+
switch (typeof obj) {
75+
case "integer":
76+
return format("%d", obj).len() + 3
77+
case "float":
78+
return 7;
79+
case "null":
80+
case "function": // Silently setting this to null
81+
return 1;
82+
case "bool":
83+
return 2;
84+
case "blob":
85+
case "string":
86+
return obj.len()+3;
87+
case "table":
88+
case "array":
89+
local size = 3;
90+
foreach ( k,v in obj ) {
91+
size += _sizeof(k) + _sizeof(v)
92+
}
93+
return size;
94+
default:
95+
throw ("Can't serialize " + typeof obj);
96+
}
97+
}
98+
99+
// Calculates an 8-bit CRC
100+
static function LRC8 (data, offset = 0) {
101+
local LRC = 0x00;
102+
for (local i = offset; i < data.len(); i++) {
103+
LRC = (LRC + data[i]) & 0xFF;
104+
}
105+
return ((LRC ^ 0xFF) + 1) & 0xFF;
106+
}
107+
108+
static function _serialize (b, obj) {
34109
switch (typeof obj) {
35110
case "integer":
36111
return _write(b, 'i', format("%d", obj));
37112
case "float":
38-
local f = format("%0.7f", obj).slice(0,9);
39-
while (f[f.len()-1] == '0') f = f.slice(0, -1);
40-
return _write(b, 'f', f);
113+
// 'F' is for new floats, 'f' is for legacy floats
114+
local bl = blob(4);
115+
bl.writen(obj, 'f');
116+
return _write(b, 'F', bl);
41117
case "null":
42118
case "function": // Silently setting this to null
43119
return _write(b, 'n');
@@ -58,13 +134,10 @@ class Serializer {
58134
return;
59135
default:
60136
throw ("Can't serialize " + typeof obj);
61-
// Utils.log("Can't serialize " + typeof obj);
62137
}
63138
}
64139

65-
66-
function _write(b, type, payload = null) {
67-
140+
static function _write(b, type, payload = null) {
68141
// Calculate the lengths
69142
local prefix_length = true;
70143
local payloadlen = 0;
@@ -94,30 +167,9 @@ class Serializer {
94167
}
95168
}
96169

97-
98-
// Deserialize a string into a variable
99-
function deserialize (s, prefix = null) {
100-
// Read and check the prefix and header
101-
local prefix_len = (prefix == null) ? 0 : prefix.len();
102-
local header_len = 3;
103-
s.seek(0);
104-
local pfx = prefix_len > 0 ? s.readblob(prefix_len) : null;
105-
local len = s.readn('w');
106-
local crc = s.readn('b');
107-
if (s.len() != len+prefix_len+header_len) throw "Expected " + len + " bytes";
108-
// Check the prefix
109-
if (prefix != null && pfx.tostring() != prefix.tostring()) throw "Prefix mismatch";
110-
// Check the CRC
111-
local _crc = LRC8(s, prefix_len+header_len);
112-
if (crc != _crc) throw format("CRC err: 0x%02x != 0x%02x", crc, _crc);
113-
// Deserialise the rest
114-
return _deserialize(s, prefix_len+header_len).val;
115-
}
116-
117-
function _deserialize (s, p = 0) {
170+
static function _deserialize (s, p = 0) {
118171
for (local i = p; i < s.len(); i++) {
119172
local t = s[i];
120-
// Utils.log("Next type: 0x%02x", t)
121173

122174
switch (t) {
123175
case 'n': // Null
@@ -127,14 +179,18 @@ class Serializer {
127179
s.seek(i+3);
128180
local val = s.readblob(len).tostring().tointeger();
129181
return { val = val, len = 3+len };
130-
case 'f': // Float
182+
case 'F': // New Floats
183+
local len = s[i+1] << 8 | s[i+2];
184+
s.seek(i+3);
185+
local val = s.readblob(len).readn('f');
186+
return { val = val, len = 3+len };
187+
case 'f': // Legacy Float deserialization :(
131188
local len = s[i+1] << 8 | s[i+2];
132189
s.seek(i+3);
133190
local val = s.readblob(len).tostring().tofloat();
134191
return { val = val, len = 3+len };
135192
case 'b': // Bool
136193
local val = s[i+1];
137-
// Utils.log("** Bool with value: %s", (val == 1) ? "true" : "false")
138194
return { val = (val == 1), len = 2 };
139195
case 'B': // Blob
140196
local len = s[i+1] << 8 | s[i+2];
@@ -150,7 +206,6 @@ class Serializer {
150206
if (len > 0) {
151207
val = s.readblob(len).tostring();
152208
}
153-
// Utils.log("** String with length %d (0x%02x 0x%02x) and value: %s", len, s[i+1], s[i+2], val)
154209
return { val = val, len = 3+len };
155210
case 't': // Table
156211
case 'a': // Array
@@ -159,14 +214,8 @@ class Serializer {
159214
i += 3;
160215
local tab = null;
161216

162-
if (t == 'a') {
163-
// Utils.log("** Array with " + nodes + " nodes");
164-
tab = [];
165-
}
166-
if (t == 't') {
167-
// Utils.log("** Table with " + nodes + " nodes");
168-
tab = {};
169-
}
217+
if (t == 'a') tab = [];
218+
if (t == 't') tab = {};
170219

171220
for (local node = 0; node < nodes; node++) {
172221

@@ -178,25 +227,13 @@ class Serializer {
178227
i += v.len;
179228
len += v.len;
180229

181-
// Utils.log("** Node %d: Key = '%s' (%d), Value = '" + v.val + "' [%s] (%d)", node, k.val, k.len, typeof v.val, v.len)
182-
183-
if (typeof tab == "array") tab.push(v.val);
184-
else tab[k.val] <- v.val;
230+
if (typeof tab == "array") tab.push(v.val);
231+
else tab[k.val] <- v.val;
185232
}
186233
return { val = tab, len = len+3 };
187234
default:
188235
throw format("Unknown type: 0x%02x at %d", t, i);
189236
}
190237
}
191238
}
192-
193-
194-
function LRC8 (data, offset = 0) {
195-
local LRC = 0x00;
196-
for (local i = offset; i < data.len(); i++) {
197-
LRC = (LRC + data[i]) & 0xFF;
198-
}
199-
return ((LRC ^ 0xFF) + 1) & 0xFF;
200-
}
201-
202239
}

0 commit comments

Comments
 (0)