cyberkitsune / pso2proxy

A protocol proxy for PSO2

🦊💛💻

pso2proxy's Issues

Reloadable block names

This way you can change them without restarting the whole server.

Error 630 when changing to vita blocks

I am able to login with my credentials and got into any block. However, when attempting to change blocks, I get an Error 630. Tried deleting blocknames.resources.json and it still happened.

A friend of mine is able to change blocks, and he didn't pull the latest version (last pull was last week according to him) and he got the blocknames.resources.json deleted as well.

Server is running Debian 7 x86, also compiled all dependencies from source.

There seems to be sth wrong with my server

I tried to connect my own server and it show like this

[ShipProxy] Opened listen socked on port 12136 for new ship.
[BlockCache] Cached new block for ship 12100, Holding onto it for 5 minutes...
[ShipProxy] Opened listen socked on port 12119 for new ship.
[BlockCache] Cached new block for ship 12100, Holding onto it for 5 minutes...
[ShipProxy] New client connected!
[ShipProxy] Client is looking for block on port 12119...
Unhandled Error
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/twisted/python/log.py", line 73, in callWithContext
return context.call({ILogContext: newCtx}, func, _args, *_kw)
File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
return self.currentContext().callWithContext(ctx, func, _args, *_kw)
File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
return func(args,*kw)
File "/usr/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite
why = selectable.doRead()
--- ---
File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py", line 1070, in doRead
protocol.makeConnection(transport)
File "/usr/lib/python2.7/dist-packages/twisted/internet/protocol.py", line 481, in makeConnection
self.connectionMade()
File "/root/PSO2Proxy/proxy/ShipProxy.py", line 189, in connectionMade
blocks.blockList[port][0], port, blocks.blockList[port][1]))
exceptions.UnicodeEncodeError: 'ascii' codec can't encode characters in position 71-80: ordinal not in range(128)

BTW I use Ubuntu 14.04

"Exit" command

Would it be possible to add an exit command to terminate the server with (for convenience)?

PSO2Proxy to Checks ships on round robin on port 12#99, just like the real PSO2

Some of the proxies having issues connecting to 210.189.208.1:12199
we should connect to 210.189.208.16:12299, etc. and handle timeouts and just check another ship's status

    Traceback (most recent call last):
      File "/usr/local/lib/python2.7/dist-packages/twisted/python/log.py", line 73, in callWithContext
        return context.call({ILogContext: newCtx}, func, *args, **kw)
      File "/usr/local/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
        return self.currentContext().callWithContext(ctx, func, *args, **kw)
      File "/usr/local/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
        return func(*args,**kw)
      File "/usr/local/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite
        why = selectable.doRead()
    --- <exception caught here> ---
      File "/usr/local/lib/python2.7/dist-packages/twisted/internet/tcp.py", line 1069, in doRead
        protocol.makeConnection(transport)
      File "/usr/local/lib/python2.7/dist-packages/twisted/internet/protocol.py", line 481, in makeConnection
        self.connectionMade()
      File "~/PSO2Proxy/proxy/queryProtocols.py", line 34, in connectionMade
        self.transport.write(ships.scrape_ship_packet("210.189.208.1", 210.189.208.1, myIpAddress))
      File "~/PSO2Proxy/proxy/data/ships.py", line 88, in scrape_ship_packet
        s.connect((ship_ip, ship_port))
      File "/usr/lib/python2.7/socket.py", line 224, in meth
        return getattr(self._sock,name)(*args)
    socket.error: [Errno 110] Connection timed out

Color and other formatting between irc and in-game globalchat.py

it would be neat if when I send a message in-game through proxy chat like "{red}RED TEXT WOOO" that when the IRC bot posts it in the IRC channel that the red text portion is properly red text instead of displaying {red}.

also the reverse would also be neat, sending a colored IRC message that is properly formatted in-game, though you would want to strip out any formatting characters that the game does not support like for bold and italic.

be sure to make it a config setting, where enabling it does the above behaviors and disableing it leaves everything alone; but possibly still strips all formatting characters from sent IRC messages. maybe 3 modes? formatting on, only strip formatting sent from IRC, and completely off?

Globalchat Mute feature

Sometimes a person is being obnoxious in chat, either through saying Terrible Things, normal spam, or the unthinkable; using Global Chat Autowords.

I feel everyone should have the option to have Global Chat enabled, but if someone is being terrible the admins should have the ability to mute them from saying anything more, maybe unmute them later, but are still able to see everyone chatting in the global chat.

Add Ship Number to GlobalChat plugin like [G-##]

Users seem to ask who are on what ship alot, let include that ship number in the proxy chat

Include in-game !help command

Setup an in-game command to list out simple documentation on available commands.

e.g.)
!help

Return:
Something something PSO2 Proxy Commands
IRC Channel: irc.badnik.net #pso2proxypublic

|count - Returns the number of people currently on your proxy

|g - Sends to the global inter-proxy chat channel

|goff - Disables and mutes the inter-proxy chat channel for you.

|gon - Enables and shows the inter-proxy chat channel for you.

!blame - Blame CyberKitsune for the DDoS Attacks

etc...

User Timeout

http://pastebin.com/z8sfpmcn

user reported that they were left hanging while trying to login, console/server log had this error waiting.

banList.txt missing in repo

Getting [Errno 2] No such file or directory: 'banList.txt' on a clean install.

Block names not updating

Since last week the english block names have stayed with the previous list and doesn't update. Is there a cache I have to delete?

http://i.imgur.com/kBuVysM.jpg

Adding install instructions for certain plugins

I just enabled the Translation plugin for my proxy by symlink-ing it.
But it didn't work at first because i didn't install the unicodescript and goslate libs.

And it seems i wasn't the only one with these kind of problems.

I solved my problem in the end by doing the following:

apt-get install python-pip | pip install unicodescript | pip install goslate

Maybe we should add those to the Readme.md or at least mention somewhere that certain plugins require more python libs and howto install those.

weird issue with g command when in pso2

http://i.imgur.com/jIOy3BV.png
this error randomly popped up and when i restarted the proxy it went away >.> was wondering if there is a way to prevent that error

Is anyone able to connect

I was connecting until 3am and when I tried to get on now I can't it gives me this error now

Users can't log in if environment running the proxy doesn't support UTF-8

            print("[ShipProxy] Found address %s for port %i, named %s" % (
                blocks.blockList[port][0], port, blocks.blockList[port][1]))

proxy/ShipProxy.py

I had some issues with my locale settings where UTF-8 wasn't enabled properly, so whenever someone tried to connect this print command failed (UnicodeEndodeError) and they got disconnected.
The solution was simply to fix my LC_ environment variables, but it seems pretty dumb that logging failing can cause disconnects.

You should probably wrap it in a try: block or something.

Note that the system supports UTF-8 but my tty settings couldn't handle utf-8 characters being printed.

[Request] Something about logged in logged out for proxy users

Excuse me, i wanna request something about pso2proxy feature/plugin.

Feature/plugin: logged in and logged out for all proxy user inside proxy through System Chat.
Using player name,
i.e. : [Proxy] %PlayerName% has logged in. / [Proxy] %PlayerName% has logged out.

Sorry if wrong session/place.
Thanks.

Major Connection Issues

I am watching my server log and people are getting 630'd left and right.

it is like they are playing musical chairs here. it seems to have started around 3pm EDT on 10/4/14, an hour before the Mine Defense 3 that was scheduled. I'd provide logs but i want to see if I can remove the IP addresses of my clients or contact me privately for them

Suppressing ProxyFactory messages in log when "noisy" is false

Any chance to have the following messages in the log supressed when the the "noisy" config option is set to false? They make up a majority of the log.

2014-08-05 06:36:49+0000 [-] Starting factory <main.ProxyFactory instance at 0x16a8fc8>
2014-08-05 06:36:49+0000 [-] ProxyFactory starting on 12810

On shared ship ONLY, Make Team Packet be handled differently in Translator.py

Did some testing. Turns out sega uses the team chat packet on shared ship for messages sent from people not in your party that are in quest with you.

Thanks sega :|

Here's a new task.

• Make Team packet be handled differently ONLY when sent from shared ship

Use this for reference to the original issue: #152

Here are some screenshots after adding the below code to translator.py:
http://i.imgur.com/H5VZ19k.jpg
http://i.imgur.com/bjyhHzx.jpg

PS the below code is not a fix its just a copy paste from 0x7,0x0 as a test

@p.PacketHook(0x7, 0x11)    
def get_chat_packet(context, packet):
    """
    :type context: ShipProxy.ShipProxy
    """
    try:
        if context.psoClient and context.playerId and data.clients.connectedClients[context.playerId].preferences.get_preference('translate_out'):
            player_id = struct.unpack_from("I", packet, 0x8)[0]
            if player_id != 0:  # ???
                return
            channel_id = struct.unpack_from("I", packet, 0x14)[0]
            message = packet[0x1C:].decode('utf-16').rstrip("\0")
            d = threads.deferToThread(generate_translated_message, player_id, channel_id, message, "ja", "en")
            d.addCallback(context.peer.send_crypto_packet)
            return None
    except KeyError:
        return packet
    if context.peer.psoClient and context.peer.playerId in data.clients.connectedClients:
        user_prefs = data.clients.connectedClients[context.peer.playerId].preferences
        if not user_prefs.get_preference('translate_chat'):
            return packet
        player_id = struct.unpack_from("I", packet, 0x8)[0]
        if player_id == 0:  # We sent it
            return packet
        channel_id = struct.unpack_from("I", packet, 0x14)[0]
        message = packet[0x1C:].decode('utf-16').rstrip("\0")
        if message.startswith("/"):
            return packet  # Command
        japanese = False
        for char in message:
            char_script = script(unicode(char))
            if char_script != 'Latin' and char_script != 'Common':
                japanese = True
                break
        if not japanese:
            return packet
        d = threads.deferToThread(generate_translated_message, player_id, channel_id, message, "en", "ja")
        d.addCallback(context.peer.send_crypto_packet)
        return None
    return packet


def generate_translated_message(player_id, channel_id, message, end_lang, start_lang):
    if provider == "Bing" and time.time() - lastKeyTime >= 600:
        translator.access_token = translator.get_access_token()

    try:
        if end_lang == "ja":
            message_string = "%s" % translator.translate(message, end_lang, start_lang)
        else:
            message_string = "%s {def}(%s)" % (translator.translate(message, end_lang, start_lang), message)
    except Exception as e:
        print (str(e))
        message_string = message

    return packetFactory.ChatPacket(player_id, message_string, channel_id).build()

What ship are they on?

the "clients" command tells you everything except what ship each client is playing on

Indentation issue

There a space at the beginning of line 97 before the tabs in the main 'proxy/PSO2Proxy.py' file that makes the python interpretor complain about inconsistend indentation and thus refusing to start the program.

Need help !

I had download Pso 2 Tweaker last week. When i login game and create character then i press play. the loading screen appear but the game has stuck in that.

I set proxy, run troubleshooting, update game, uninstall and reinstall game but not change.

Please help me guy !

Sorry, my english isn't good

Translate.py

Just tried to use the Translate.py plugin to find "no module named unicodescript"
Not sure if there is something I need to install or if there is an issue with the translate plugin.

Screenshot:
http://puu.sh/bqLAQ/1330c85f54.png

The data buffer read while loop can get stuck

The ShipProxy read loop can get stuck if a packet reports an invalid length. (Zero or negative. ) This will cause the proxy to lock up.

None-Type's and 630's at login

Upon First login users are 630'd and then are able to login upon the second login are allowed on. New None-Type users are generated. As seen in the images

Clients remaining connected to the proxy but freezing.

It looks like, randomly, whenever someone is trying to get the ship status the proxy is causing the clients to disconnect from the SEGA servers. The clients are remaining connected to the proxy, and do not get any error messages in-game, but are effectively frozen.
This issue has happened again after a proxy and server restart, and has happened twice so far.

The error messages, as taken from the log files, are here:
http://pastebin.com/KeGvCMn0
http://pastebin.com/GWjuZMmT

globalchat.py autoexec runs too early

[GlobalChat] IRC Connected!
[IRC] [NOTICE] irc.irc.irc *** Looking up your hostname...
[IRC] [NOTICE] irc.irc.irc *** Checking ident...
[IRC] [NOTICE] irc.irc.irc *** No ident response; username prefixed with ~
[IRC] [NOTICE] irc.irc.irc *** Couldn't resolve your hostname; using your IP address instead
[IRC-AUTO] >>> /msg NickServ IDENTIFY NOTREALLYTHEPASSWORD
[GlobalChat] Joined #GlobalProxyChat
[IRC] [NOTICE] Global![email protected] [Logon News - Mar 04 2010] Follow IRC on Twitter! http://twitter.com/irc - Now looking for feedback on how we can improve the user experience. Tweet us your ideas!
[IRC] [NOTICE] NickServ![email protected] This nickname is registered and protected. If it is your
[IRC] [NOTICE] NickServ![email protected] nick, type /msg NickServ IDENTIFY password. Otherwise,
[IRC] [NOTICE] NickServ![email protected] please choose a different nick.

looks like my identify command is not going off when it should be, and I can't seem to manually send the /msg command properly either.

if I do have the power to do this then that just means somebody is lacking in their documentation

unhandled error

I honestly don't know where else to put this, but I saw someone trip an error in the console so here is a paste-dump of it and the console entries around it: http://pastebin.com/9CrA5ZzB

I censored out anything I did not want to post, but if you need some of that data I can send it in a more private fashion.

Sometimes a player isn't always registered when they initially change blocks.

SEGA ID will appear as None, causes disconnect.

if you forget # in channel name in cfg it causes all types of problems

recommend making it so that it errors and doesnt join if you forget # in the beginning of the channel name otherwise messages wont be sent back and forth and everything will be name + host

yes im talking about gchat irc

Proxy Usage for SEA?

Hi there, I noticed PlayPark has a PSO2 English Client and was wondering if your proxy services work with the SEA servers?

If so, how do I set that up?

PSO2Proxy uses way too many sockets.

This is a bug. The protocol normally uses ~999 TCP ports but technically could be made way more efficient.

This ticket will stay open until a new method of handling this is implemented. Feel free to help brainstorm in the comments.

UnicodeEncodeError while trying to log in

After grabbing the latest 4 commits, I can no longer log in. I get the following output after entering my username and password:

/usr/lib64/python2.6/site-packages/twisted/internet/endpoints.py:30: DeprecationWarning: twisted.internet.interfaces.IStreamClientEndpointStringParser was deprecated in Twisted 14.0.0: This interface has been superseded by IStreamClientEndpointStringParserWithReactor.
  from twisted.internet.interfaces import (
/usr/lib64/python2.6/site-packages/Crypto/Util/number.py:57: PowmInsecureWarning: Not using mpz_powm_sec.  You should rebuild using libgmp >= 5 to avoid timing attack vulnerability.
  _warn("Not using mpz_powm_sec.  You should rebuild using libgmp >= 5 to avoid timing attack vulnerability.", PowmInsecureWarning)
Unhandled Error
Traceback (most recent call last):
  File "/usr/lib64/python2.6/site-packages/twisted/python/log.py", line 73, in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "/usr/lib64/python2.6/site-packages/twisted/python/context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib64/python2.6/site-packages/twisted/python/context.py", line 81, in callWithContext
    return func(*args,**kw)
  File "/usr/lib64/python2.6/site-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite
    why = selectable.doRead()
--- <exception caught here> ---
  File "/usr/lib64/python2.6/site-packages/twisted/internet/tcp.py", line 1069, in doRead
    protocol.makeConnection(transport)
  File "/usr/lib64/python2.6/site-packages/twisted/internet/protocol.py", line 481, in makeConnection
    self.connectionMade()
  File "/home/ec2-user/PSO2Proxy/proxy/ShipProxy.py", line 186, in connectionMade
    blocks.blockList[port][0], port, blocks.blockList[port][1]))
exceptions.UnicodeEncodeError: 'ascii' codec can't encode characters in position 69-77: ordinal not in range(128)

I am running this from an Amazon Linux AMI system.

EQ Notice for ship02 only?

Hi, I need some help with my pso2proxy setup. It's not an issue but a query

Can I set EQ_Notice to point other ship eq notice (eg. ship09) because sometimes other ship have different EQ time.

Thanks in advance

Proxy server Public/Private RSA keys size

I noticed upon reinstalling the PSOProxy server that it can't use an RSA Public/Private pair of keys when the key size exceed 1024 bits. Is this a limitation of the original client that doesn't support public keys of higher size or is this on PSO2Proxy side ?

Just curious as I went almost crazy not understanding why I got "Server timed out" message and have been messing with keys for an hour and I just realised this.

EQ Noitce to IRC code is broken

Unhandled error in Deferred:
Unhandled Error
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/twisted/web/_newclient.py", line 1154, in _bodyDataFinished_CONNECTED
self._bodyProtocol.connectionLost(reason)
File "/usr/lib/python2.7/dist-packages/twisted/web/client.py", line 1964, in connectionLost
self.deferred.callback(b''.join(self.dataBuffer))
File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 382, in callback
self._startRunCallbacks(result)
File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
self._runCallbacks()
--- ---
File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks
current.result = callback(current.result, _args, *_kw)
File "/home/alam/git/PSO2Proxy/proxy/plugins/EQ_Notice.py", line 193, in EQBody
GlobalChat.ircBot.send_channel_message("[EQ Notice Ship %02d] Incoming EQ Report from PSO2es: %s" % (ship + 1, msg_eq[ship]))
File "/home/alam/git/PSO2Proxy/proxy/plugins/GlobalChat.py", line 94, in send_channel_message
self.msg(self.factory.channel, message)
File "/usr/lib/python2.7/dist-packages/twisted/words/protocols/irc.py", line 1613, in msg
self.sendLine(fmt + line)
File "/usr/lib/python2.7/dist-packages/twisted/words/protocols/irc.py", line 1099, in sendLine
self._reallySendLine(line)
File "/usr/lib/python2.7/dist-packages/twisted/words/protocols/irc.py", line 1095, in _reallySendLine
return basic.LineReceiver.sendLine(self, lowQuote(line) + '\r')
File "/usr/lib/python2.7/dist-packages/twisted/protocols/basic.py", line 635, in sendLine
return self.transport.write(line + self.delimiter)
File "/usr/lib/python2.7/dist-packages/twisted/internet/_newtls.py", line 191, in write
FileDescriptor.write(self, bytes)
File "/usr/lib/python2.7/dist-packages/twisted/internet/abstract.py", line 348, in write
raise TypeError("Data must not be unicode")
exceptions.TypeError: Data must not be unicode

Incoming japanese chat messages are not getting translated in Challenge Quests

Hello, this issue bugged me already for some time.
I could finally take a closer look at this. First I tought it is a problem with the configuration of my system, but this doesn't seem to be the case. I tested it just now on the public PSO2Proxy.

And here is the result:

Kurumi was at first not in my party then I invited her an let her say the same "Hello" message in japanese.

It seems like translation only works for people in your party, but not for nearby players.
I tested this on both a private/local (Google/Goslate) and the public (Bing/Microsofttranslator) PSO2Proxy.
Neither seems to work. Can somebody maybe look into this?

Server connection timed out / Could not decrypt RSA for client

I'm running a PSO2Proxy instance on a VPS but it stopped working after today's maintenance.

In the client, the error message Server connection timed out is shown when attempting to sign in.
On the server, the following error message is shown:

[KeyPacket] Could not decrypt RSA for client <ip>, Perhaps their client's key is unmodified? Hanging up.

I'm running through the PSO2 Tweaker with the exact same steps that have been working before.

Chosing proxy chat color

Would involve researching if party / all / team chat packet uses the same structure.

Admin commands throw exceptions

File "/PSO2Proxy/proxy/config.py", line 81, in is_admin
if sega_id in globalConfig.get_key('admins'):
exceptions.TypeError: argument of type 'NoneType' is not iterable

The error is that using the "op" console command when the admin list is empty, it adds a null entry. Using "op" after that throws the above exception. Same problem with using deop on the last user in the list.

When having a null entry and a user tries to use an admin chat command, they get disconnected.

Need some transparency on the RSAKeyDumper DLL

Hey, just like to say great work with the server!

I've have been checking the project out and decided to setup up my own server. I'm having one issue though. I have no idea where to place the .dll files that are in /tools.

[PSO2Tweaker method]
"To use this with PSO2Tweaker, enable the Item Translation patch and rename the dll to rsainject.dll."

I am not exactly sure what folder the RSAKeyDumper DLL to be in?
I did renamed it to "rsainject.dll" before hand.
I have tried placing it directly next to PSO2.exe and PSOTweaker.exe, but no files are dumped?

Thanks for the support, hope to hear back soon. :)

Emergency Quest API

Error 630 on Featured Quest block, Time Attacks block, and leaving Shared Ship

Possible that leaving Shared Ship ended up directing me to one of those two blocks without my input. After the error takes me back to the ship screen, I am unable to log in again. After shutting down the game and restarting, it is still unable to connect once, but finally connects after that.

"Server connection timed out" when logging in

After the latest PSO2 update, and after grabbing the latest commit and restarting my instance of PSO2Proxy, I am no longer able to connect using my instance of the proxy. PSO2 Tweaker sees that the server is online, and the game sees that all ships are online, but when I enter my username and password to log in, I always get a "Server connection timed out" error.

The public instance of the proxy works fine. I was able to log in with the public one, but I switched back to my private instance afterwards and it still would not connect.

When I attempt to log in, the proxy outputs the following:

[ShipStatus] Client connected IPv4Address(TCP, '<my_ip_address>', 14152)! Sending ship list packet...
[BlockQuery] <my_ip_address>:14158 wants to load-balance on port 12200!
[ShipProxy] New client connected!
[ShipProxy] Client is looking for block on port 12236...
[ShipProxy] Found address 210.189.208.22 for port 12236, named B-036:カジノ推奨
[ShipProxy] Connected to block server!
[ShipProxy] Client at <my_ip_address> lost connection.

Team chat translated packet might not be formatted correctly

http://imgur.com/B7Eya1g,QmjZDZX,5uIffYv

Pretty sure everyone in team chat can see the message though as they responded the first time i posted it before i decided to take a screenshot.

Further Testing should be done.

Allow configurable amount of ships.

Temporary workaround for bug #30.

ChatTranslator.py plugin not working on Shared Ship

http://pastebin.com/LT7caxmZ

http://pastebin.com/VBS93Gyr (full log)

the main issue is that the translation is failing because it's 503'ing meaning it's timing out

any idea what could be causing this?

• find reason for issues with translation of chat messages (#151 - similar but yeah these errors might be the cause)

an unhandled error (will look into) caused by dead irc server

Web "RCON" API

Proxy load rules

It seems we have an user, HyperAmy7, from IP 99.59.4.126, that can connect the JP server fennec? but not US server arcticfox?, it just disconnect them, for now, he is using my proxy server, it would be nice to have a set of rules we can setup to know when not to send users to proxy server that would just disconnect them