gaynetdinov / ex_marshal Goto Github PK
View Code? Open in Web Editor NEWRuby Marshal format implemented in Elixir
License: ISC License
ex_marshal's People
Contributors
Stargazers
Watchers
Forkers
ricardohsd mgwidmann hisea simonbowen gialib sktocha eole-repos luisgabrielroldan edmondfrank paulnicholson kianmeng frerich holdsportex_marshal's Issues
symbol 'o' not supported.
I'm using this lib to decode rails 3 sessions.
It seems that ExMarshal doesnt support ruby object which Rails 3.2 puts object in flash.
i.e.
in Ruby
cookie = "BAh7CUkiD3Nlc3Npb25faWQGOgZFVEkiJTRjODRkMzUzMTFkNTc2YWUwYjVkMmNjZjRhNjY4YzY2BjsAVEkiE3VzZXJfcmV0dXJuX3RvBjsAVCIGL0kiEF9jc3JmX3Rva2VuBjsARkkiMWVlQkRhOThqT2F2Q2dkTFRSemZkM2lpMTU4Ly9JckUxVEJrY1lwZVgwQnM9BjsARkkiCmZsYXNoBjsAVG86JUFjdGlvbkRpc3BhdGNoOjpGbGFzaDo6Rmxhc2hIYXNoCToKQHVzZWRvOghTZXQGOgpAaGFzaH0GOgphbGVydFRGOgxAY2xvc2VkRjoNQGZsYXNoZXN7BjsKSSI2WW91IG5lZWQgdG8gc2lnbiBpbiBvciBzaWduIHVwIGJlZm9yZSBjb250aW51aW5nLgY7AFQ6CUBub3cw"
b= Base64.decode64(cookie)
=> "\x04\b{\tI\"\x0Fsession_id\x06:\x06ETI\"%4c84d35311d576ae0b5d2ccf4a668c66\x06;\x00TI\"\x13user_return_to\x06;\x00T\"\x06/I\"\x10_csrf_token\x06;\x00FI\"1eeBDa98jOavCgdLTRzfd3ii158//IrE1TBkcYpeX0Bs=\x06;\x00FI\"\nflash\x06;\x00To:%ActionDispatch::Flash::FlashHash\t:\n@usedo:\bSet\x06:\n@hash}\x06:\nalertTF:\f@closedF:\r@flashes{\x06;\nI\"6You need to sign in or sign up before continuing.\x06;\x00T:\t@now0"
Marshal.load b
=> {"session_id"=>"4c84d35311d576ae0b5d2ccf4a668c66",
"user_return_to"=>"/",
"_csrf_token"=>"eeBDa98jOavCgdLTRzfd3ii158//IrE1TBkcYpeX0Bs=",
"flash"=>
#<ActionDispatch::Flash::FlashHash:0x007fe33510dc70
@closed=false,
@flashes={:alert=>"You need to sign in or sign up before continuing."},
@now=nil,
@used=#<Set: {:alert}>>}
Where in ExMarshal I get:
** (ExMarshal.DecodeError) term which starts with the following symbol is not supported: "o" lib/ex_marshal/decoder.ex:39: ExMarshal.Decoder.decode_element/2
lib/ex_marshal/decoder.ex:244: ExMarshal.Decoder.decode_hash/4
lib/ex_marshal/decoder.ex:3: ExMarshal.Decoder.decode/1
Encode/decode Float Infinity
Log the whole unsupported data structure
I'm currently getting this error with plug_rails_session
(ExMarshal.Errors.DecodeError term which starts with the following symbol is not supported: "o")
It would be nice to have the whole data structure so I can decode it in ruby and track down where this error may come from ?
No match of right hand side value
Given this session:
{
"session_id"=>"a8170a95eb2f56f046da8be50d92e8a5",
"s3_video_region"=>"us-west-1",
"admin_return_to"=>"/gizmo/reports/distribution_receipts",
"user_return_to"=>"http://usertesting.dev/admins/auth/google_oauth2/callback?state=eec87db6b1eaf789d869c6ad7def175d6d50240060b95f24&code=4/ItZbxW-FxI0TlRQelK0N-cWcUdPl1NzQIRTrI6H3AI0",
"warden.user.admin.key"=>[[195], nil],
"warden.user.admin.session"=>{"last_request_at"=>1476311578},
"_csrf_token"=>"K+aJ46j1ZJxCgPVcxMoWQTVS+0OOKeH4NCOsvKpE+Hg=",
"foo"=>"bar"
}I get this charlist:
<<4, 8, 123, 13, 73, 34, 15, 115, 101, 115, 115, 105, 111, 110, 95, 105, 100, 6,
58, 6, 69, 84, 73, 34, 37, 97, 56, 49, 55, 48, 97, 57, 53, 101, 98, 50, 102,
53, 54, 102, 48, 52, 54, 100, 97, 56, 98, 101, 53, 48, 100, 57, 50, 101, 56,
97, 53, 6, 59, 0, 84, 73, 34, 20, 115, 51, 95, 118, 105, 100, 101, 111, 95,
114, 101, 103, 105, 111, 110, 6, 59, 0, 84, 73, 34, 14, 117, 115, 45, 119,
101, 115, 116, 45, 49, 6, 59, 0, 84, 73, 34, 20, 97, 100, 109, 105, 110, 95,
114, 101, 116, 117, 114, 110, 95, 116, 111, 6, 59, 0, 84, 73, 34, 41, 47, 103,
105, 122, 109, 111, 47, 114, 101, 112, 111, 114, 116, 115, 47, 100, 105, 115,
116, 114, 105, 98, 117, 116, 105, 111, 110, 95, 114, 101, 99, 101, 105, 112,
116, 115, 6, 59, 0, 70, 73, 34, 19, 117, 115, 101, 114, 95, 114, 101, 116,
117, 114, 110, 95, 116, 111, 6, 59, 0, 70, 73, 34, 1, 163, 104, 116, 116, 112,
58, 47, 47, 117, 115, 101, 114, 116, 101, 115, 116, 105, 110, 103, 46, 100,
101, 118, 47, 97, 100, 109, 105, 110, 115, 47, 97, 117, 116, 104, 47, 103,
111, 111, 103, 108, 101, 95, 111, 97, 117, 116, 104, 50, 47, 99, 97, 108, 108,
98, 97, 99, 107, 63, 115, 116, 97, 116, 101, 61, 101, 101, 99, 56, 55, 100,
98, 54, 98, 49, 101, 97, 102, 55, 56, 57, 100, 56, 54, 57, 99, 54, 97, 100,
55, 100, 101, 102, 49, 55, 53, 100, 54, 100, 53, 48, 50, 52, 48, 48, 54, 48,
98, 57, 53, 102, 50, 52, 38, 99, 111, 100, 101, 61, 52, 47, 73, 116, 90, 98,
120, 87, 45, 70, 120, 73, 48, 84, 108, 82, 81, 101, 108, 75, 48, 78, 45, 99,
87, 99, 85, 100, 80, 108, 49, 78, 122, 81, 73, 82, 84, 114, 73, 54, 72, 51,
65, 73, 48, 6, 59, 0, 84, 73, 34, 26, 119, 97, 114, 100, 101, 110, 46, 117,
115, 101, 114, 46, 97, 100, 109, 105, 110, 46, 107, 101, 121, 6, 59, 0, 84,
91, 7, 91, 6, 105, 1, 195, 48, 73, 34, 30, 119, 97, 114, 100, 101, 110, 46,
117, 115, 101, 114, 46, 97, 100, 109, 105, 110, 46, 115, 101, 115, 115, 105,
111, 110, 6, 59, 0, 84, 123, 6, 73, 34, 20, 108, 97, 115, 116, 95, 114, 101,
113, 117, 101, 115, 116, 95, 97, 116, 6, 59, 0, 84, 108, 43, 7, 76, 145, 254,
87, 73, 34, 16, 95, 99, 115, 114, 102, 95, 116, 111, 107, 101, 110, 6, 59, 0,
70, 73, 34, 49, 75, 43, 97, 74, 52, 54, 106, 49, 90, 74, 120, 67, 103, 80, 86,
99, 120, 77, 111, 87, 81, 84, 86, 83, 43, 48, 79, 79, 75, 101, 72, 52, 78, 67,
79, 115, 118, 75, 112, 69, 43, 72, 103, 61, 6, 59, 0, 70, 73, 34, 8, 102, 111,
111, 6, 59, 0, 84, 73, 34, 8, 98, 97, 114, 6, 59, 0, 84>>and this error:
** (MatchError) no match of right hand side value: ""
lib/ex_marshal/decoder.ex:57: ExMarshal.Decoder.decode_fixnum/2
lib/ex_marshal/decoder.ex:110: ExMarshal.Decoder.decode_string/2
lib/ex_marshal/decoder.ex:244: ExMarshal.Decoder.decode_hash/4
lib/ex_marshal/decoder.ex:3: ExMarshal.Decoder.decode/1It's crashing somewhere parsing the long google oauth2 callback url
Decoding bug: nested atom wrongfully parsed to nil
Hello, when working to decrypt and decode Rails 4.0 sessions, I stumbled upon a bug. Long story short, get in iex and follow this:
marshalled = "\x04\b{\tI\"\x0Fsession_id\x06:\x06ETI\"%9abcec93dd746529e91623edaa51a69f\x06;\x00TI\"\x1Fwarden.user.spree_user.key\x06;\x00T[\a[\x06i\x02\xC0{I\"\x19bvxmDzqkuhesXSxn85_6\x06;\x00TI\"\nflash\x06;\x00T{\aI\"\fdiscard\x06;\x00T[\x06:\fsuccessI\"\fflashes\x06;\x00T{\x06;\x06I\"\eLogged in successfully\x06;\x00TI\"\x10_csrf_token\x06;\x00FI\"1RlmOff4/q/+Pb2JhNObj1uJkPe7vRwi/BOHB34NYZ84=\x06;\x00F"
actual = ExMarshal.decode(marshalled)
=> %{"_csrf_token" => "RlmOff4/q/+Pb2JhNObj1uJkPe7vRwi/BOHB34NYZ84=",
"flash" => %{"discard" => [:success],
"flashes" => %{nil: "Logged in successfully"}},
"session_id" => "9abcec93dd746529e91623edaa51a69f",
"warden.user.spree_user.key" => [[31680], "bvxmDzqkuhesXSxn85_6"]}
# This has been obtained via Ruby code: Marshal.load(marshalled), copied in Elixir (added % signs for the maps) and then printed to Elixir console.
expected = %{"_csrf_token" => "RlmOff4/q/+Pb2JhNObj1uJkPe7vRwi/BOHB34NYZ84=",
"flash" => %{"discard" => [:success],
"flashes" => %{success: "Logged in successfully"}},
"session_id" => "9abcec93dd746529e91623edaa51a69f",
"warden.user.spree_user.key" => [[31680], "bvxmDzqkuhesXSxn85_6"]}
get_in expected, ["flash", "flashes"]
=> %{success: "Logged in successfully"}
# This is the problem:
get_in actual, ["flash", "flashes"]
=> %{nil: "Logged in successfully"}It seems that this success key in the nested map is not parsed properly? The problem persists regardless of the nullify_objects config option, too.
Decoding error
I'm trying to decode a rails session that is dumped with Marshal
2.1.5 :032 > s.data.data
=> "\x04\b{\tI"\x19warden.user.user.key\x06:\x06ET[\a[\x06I"\x1D548814a064657654501a0000\x06;\x00TI""$2a$10$73PE6lJ5iYxwumwZQ/qjjO\x06;\x00TI"\x1Dwarden.user.user.session\x06;\x00T{\x06I"\x16unique_session_id\x06;\x00TI"\x19AsqZJFimQjSEsYyAgL8T\x06;\x00FI"\nflash\x06;\x00T{\aI"\fdiscard\x06;\x00T[\x06I"\vnotice\x06;\x00FI"\fflashes\x06;\x00T{\x06@\x13I" Je bent succesvol ingelogd.\x06;\x00TI"\x10_csrf_token\x06;\x00FI"1aOaum1ZeInfFvRgG8srczhiGnbpx41VIdix7+4QKxzo=\x06;\x00F"
2.1.5 :033 > Marshal.load(s.data.data)
=> {"warden.user.user.key"=>[["548814a064657654501a0000"], "$2a$10$73PE6lJ5iYxwumwZQ/qjjO"], "warden.user.user.session"=>{"unique_session_id"=>"AsqZJFimQjSEsYyAgL8T"}, "flash"=>{"discard"=>["notice"], "flashes"=>{"notice"=>"Je bent succesvol ingelogd."}}, "_csrf_token"=>"aOaum1ZeInfFvRgG8srczhiGnbpx41VIdix7+4QKxzo="}
When i decode this in elixir i get:
ExMarshal.decode(session.data)
** (CaseClauseError) no case clause matching: "@"
lib/ex_marshal/decoder.ex:10: ExMarshal.Decoder.decode_element/2
lib/ex_marshal/decoder.ex:204: ExMarshal.Decoder.decode_hash/4
lib/ex_marshal/decoder.ex:205: ExMarshal.Decoder.decode_hash/4
lib/ex_marshal/decoder.ex:3: ExMarshal.Decoder.decode/1
Any idea how to solve this ?
Issues with decoding rails 3.2 cookie - only string ivars are supported
I'm trying to use ExMarshal to decode the contents of a cookie set by rails. This is the ruby structure that is stored in the cookie in a marshal format.
{"session_id"=>"c0a56012635b795b0f6bd6a522fe3fae",
"after_sign_up_redirect_to"=>"/",
"after_sign_in_redirect_to"=>"/",
"omniauth.is_abode"=>false,
"warden.user.user.key"=>[[70989], "$2a$10$D4Msl4T4JHfkFOQHi0fKTe"],
"warden.user.user.session"=>{"last_request_at"=>2017-02-22 20:44:16 UTC},
"foo"=>"bar",
"_csrf_token"=>"QUBPx+eyL2deFkAo3pA+xxMZmVso7hfLutVJlJoT8kg="}The base64 of the Marshal object looks like..
BAh7DUkiD3Nlc3Npb25faWQGOgZFVEkiJWMwYTU2MDEyNjM1Yjc5NWIwZjZiZDZhNTIyZmUzZmFlBjsAVEkiHmFmdGVyX3NpZ25fdXBfcmVkaXJlY3RfdG8GOwBGSSIGLwY7AFRJIh5hZnRlcl9zaWduX2luX3JlZGlyZWN0X3RvBjsARkkiBi8GOwBUSSIWb21uaWF1dGguaXNfYWJvZGUGOwBURkkiGXdhcmRlbi51c2VyLnVzZXIua2V5BjsAVFsHWwZpA00VAUkiIiQyYSQxMCRENE1zbDRUNEpIZmtGT1FIaTBmS1RlBjsAVEkiHXdhcmRlbi51c2VyLnVzZXIuc2Vzc2lvbgY7AFR7BkkiFGxhc3RfcmVxdWVzdF9hdAY7AFRJdToJVGltZQ3URh3A2pgMsQY6CXpvbmVJIghVVEMGOwBGSSIIZm9vBjsARkkiCGJhcgY7AFRJIhBfY3NyZl90b2tlbgY7AEZJIjFRVUJQeCtleUwyZGVGa0FvM3BBK3h4TVptVnNvN2hmTHV0VkpsSm9UOGtnPQY7AEY=
I'm then decoding this on the Elixir end with
iex> {:ok, r} = Base.decode64("BAh7DUkiD3Nlc3Npb25faWQGOgZFVEkiJWMwYTU2MDEyNjM1Yjc5NWIwZjZiZDZhNTIyZmUzZmFlBjsAVEkiHmFmdGVyX3NpZ25fdXBfcmVkaXJlY3RfdG8GOwBGSSIGLwY7AFRJIh5hZnRlcl9zaWduX2luX3JlZGlyZWN0X3RvBjsARkkiBi8GOwBUSSIWb21uaWF1dGguaXNfYWJvZGUGOwBURkkiGXdhcmRlbi51c2VyLnVzZXIua2V5BjsAVFsHWwZpA00VAUkiIiQyYSQxMCRENE1zbDRUNEpIZmtGT1FIaTBmS1RlBjsAVEkiHXdhcmRlbi51c2VyLnVzZXIuc2Vzc2lvbgY7AFR7BkkiFGxhc3RfcmVxdWVzdF9hdAY7AFRJdToJVGltZQ3URh3A2pgMsQY6CXpvbmVJIghVVEMGOwBGSSIIZm9vBjsARkkiCGJhcgY7AFRJIhBfY3NyZl90b2tlbgY7AEZJIjFRVUJQeCtleUwyZGVGa0FvM3BBK3h4TVptVnNvN2hmTHV0VkpsSm9UOGtnPQY7AEY=")
iex> ExMarshal.decode(r)However I get this error
** (ExMarshal.DecodeError) only string ivars are supported: <<58, 9, 84, 105, 109, 101, 13, 212, 70, 29, 192, 218, 152, 12, 177, 6, 58, 9, 122, 111, 110, 101, 73, 34, 8, 85, 84, 67, 6, 59, 0, 70, 73, 34, 8, 102, 111, 111, 6, 59, 0, 70, 73, 34, 8, 98, 97, 114, 6, 59, ...>> lib/ex_marshal/decoder.ex:109: ExMarshal.Decoder.decode_ivar/2
lib/ex_marshal/decoder.ex:248: ExMarshal.Decoder.decode_hash/4
lib/ex_marshal/decoder.ex:4: ExMarshal.Decoder.decode/1
(stdlib) erl_eval.erl:670: :erl_eval.do_apply/6
(iex) lib/iex/evaluator.ex:182: IEx.Evaluator.handle_eval/6
(iex) lib/iex/evaluator.ex:175: IEx.Evaluator.do_eval/4
(iex) lib/iex/evaluator.ex:155: IEx.Evaluator.eval/4
(iex) lib/iex/evaluator.ex:61: IEx.Evaluator.loop/3
(iex) lib/iex/evaluator.ex:21: IEx.Evaluator.init/4
(iex) lib/iex.ex:471: IEx.pry/3
(infabode_analytics) web/controllers/dashboard_controller.ex:9: InfabodeAnalytics.DashboardController.index/2
(infabode_analytics) web/controllers/dashboard_controller.ex:1: InfabodeAnalytics.DashboardController.action/2
(infabode_analytics) web/controllers/dashboard_controller.ex:1: InfabodeAnalytics.DashboardController.phoenix_controller_pipeline/2
(infabode_analytics) lib/infabode_analytics/endpoint.ex:1: InfabodeAnalytics.Endpoint.instrument/4
(infabode_analytics) lib/phoenix/router.ex:261: InfabodeAnalytics.Router.dispatch/2
(infabode_analytics) web/router.ex:1: InfabodeAnalytics.Router.do_call/2
(infabode_analytics) lib/infabode_analytics/endpoint.ex:1: InfabodeAnalytics.Endpoint.phoenix_pipeline/1
(infabode_analytics) lib/plug/debugger.ex:123: InfabodeAnalytics.Endpoint."call (overridable 3)"/2
I'm at a bit of loss since I've been manually creating objects in the ruby console encoding them with marshal and then base64 encoding it. Copying the base64 string across to the Elixir console, base64 decoding and decoding with ExMarshal and it works.
Is there something in my cookie structure that is causing ExMarshal to throw this error?
Term which starts with the following symbol is not supported: <<141>>
Hi 👋!
I'm using ex_marshal along with memcachex to read from a Memcached cache, shared with a Rails application. In this cache, there are records that have a field with a long avatar url, and when that field value exceeds a certain length it throws the following error:
** (ExMarshal.Errors.DecodeError) term which starts with the following symbol is not supported: <<141>>
(ex_marshal) lib/ex_marshal/decoder.ex:63: ExMarshal.Decoder.decode_element/2
(ex_marshal) lib/ex_marshal/decoder.ex:10: ExMarshal.Decoder.decode/1
In these cases, if I update the record from Rails with a shorter value, it works perfectly fine. What might be happening? Could it be something related to #10?
Thank you very much in advance :)
Use Protocols in Encoder
Update Readme to include PlugSessionRedis
Thought you should be aware of https://github.com/aposto/plug_session_redis#custom-serializers I added recently...
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.