Sorry If I'm being rather harsh here but what's the point in mapping if you can't do anything with the mapping?
I love the project I just see doing mapping now pointless as it has no feature to take the mapping anywhere.
It's a legitimate question. It's not obvious to tell what's going on from just looking at Enigma's GUI. Right now, Enigma lacks the ability to create a working source tree for Minecraft. I'm actually working on implementing that now. The eventual goal is to be able to pick a Minecraft jar, pick a mappings file, click a button, and export a fully-compilable source tree for Minecraft.
support for interfaces: "Show Interfaces" command in editor. Method rename now renames all methods related by interfaces as well as related by inheritance.
source export function: write all deobfuscated source files to a folder. The exported source doesn't compile yet. I'm working on that...
EDIT: v0.4.1 beta is out! Fixed bug with sort order when writing mappings.
Alright, so this is obviously a pre-requisite for M3 API/MPI/Loader and you are moving away from MCP. Veeerrrry interesting
Last I used Procyon though, it had a lot of problems. Granted I was using it to analyze reverse-compiled Dalvik executables, but the conversion from dex <> jar is a lossless process and the AOSP does indeed use javac. Maybe Androids' use of Java is a bit obscure to traditional Java and Procyon just isn't mature enough to support it yet; but jd-gui was most often more accurate. How far as Procyon come along? Do you contribute to it?
I have a vague question, if you don't mind. Does MCP completely decompile and deobfuscate Minecraft into working code? Judging from what you see when making mods, this seems to be the case - but I seriously find this hard to believe. What's the story there, is that true (and is that what's required for your decompiler/reverse-compiler to get to)?
The eventual goal is to be able to pick a Minecraft jar, pick a mappings file, click a button, and export a fully-compilable source tree for Minecraft.
Christ, you're crazy. Crazy brilliant xD
I'll hve to try Enigma out with my real-job reverse engineering tasks. If it turns out well, I may very well contribute via monetary means. And I don't mean have-a-beer-on-me donations; reverse engineering is hard work and good money
Alright, so this is obviously a pre-requisite for M3 API/MPI/Loader and you are moving away from MCP. Veeerrrry interesting
Last I used Procyon though, it had a lot of problems. Granted I was using it to analyze reverse-compiled Dalvik executables, but the conversion from dex <> jar is a lossless process and the AOSP does indeed use javac. Maybe Androids' use of Java is a bit obscure to traditional Java and Procyon just isn't mature enough to support it yet; but jd-gui was most often more accurate. How far as Procyon come along? Do you contribute to it?
I have a vague question, if you don't mind. Does MCP completely decompile and deobfuscate Minecraft into working code? Judging from what you see when making mods, this seems to be the case - but I seriously find this hard to believe. What's the story there, is that true (and is that what's required for your decompiler/reverse-compiler to get to)?
EDIT: Nevermind, I see this:
Christ, you're crazy. Crazy brilliant xD
I'll hve to try Enigma out with my real-job reverse engineering tasks. If it turns out well, I may very well contribute via monetary means. And I don't mean have-a-beer-on-me donations; reverse engineering is hard work and good money
Procyon has been amazing to work with. The project seems very mature now and it emits good source code. I've run into a few technical issues with Procyon, but since it's open source, I can see what's going on. Also, Procyon's maintainer has been super helpful and is just generally pleasant to work with. I would contribute to Procyon, but it hasn't been necessary yet.
I tried jd before I found Procyon, but it was just garbage. js is too dated to be of any use for this work.
MCP completely decompiles and de-obfuscates Minecraft jars in to a working source tree. It does a pretty good job of it too, so it's a tough competitor to beat. =)
And yes, reverse engineering is pretty challenging, but I think it's also a lot of fun. Enigma/M3L is a hobby project for me. It's a nice easy break from my actual job.
Feel free to use Enigma at work if you actually find it useful there. I never thought someone outside the Minecraft community would find it useful though, so that's kind of a nice surprise. Contributions are always welcome too. I have a bitcoin address, or there's always Patreon.
In other news, the 1.8-pre1 mappings have been converted to 1.8-pre3.
Wait, what?!
Yes, that's right. We have a working tool now that will (mostly) automate the conversion of mappings between different versions of Minecraft. Expect that Enigma will be able to keep up with Minecraft snapshots now. Imagine being able to test out your mods on new Minecraft snapshots... =)
Ugh... forum ruined my quote tags, but whatever you'll understand if I remove them.
Yeah well, reverse engineering in a job role is not a very common thing (and it's not the only thing I do, other roles are minor Java development, main role is embedded Linux/kernel work). I work with a lot of Chinese developers who, for whatever reason, don't agree to sharing source code but are fine with us reverse engineering to customize instead (makes no sense but whatever). Enigma would have a lot of popularity among the Android hobbyist/hacking scene such as XDA for example. As you said, jd produces garbage - there are other decompilers I've tried with varying degrees of success (nested classes and synchronized clauses will often fail to decompile in anything I've used), and of course that lack of Java > 5 support as the Procyon page says.
In other news, the 1.8-pre1 mappings have been converted to 1.8-pre3.
Wait, what?!
Yes, that's right. We have a working tool now that will (mostly) automate the conversion of mappings between different versions of Minecraft. Expect that Enigma will be able to keep up with Minecraft snapshots now. Imagine being able to test out your mods on new Minecraft snapshots... =)
Very nice! I'm super interested in helping out with development of Enigma/M3L, but I really don't know if I'll have the time to invest anything worthwhile plus I am still quite new to Minecraft modding and by no means an expert on Forge modding even. But even the fact that Enigma has deobfuscation that I can actually (and easily) fix makes it far more worthwhile to use than Forge, however we're still a quite a distance from getting a full blown API/loader anyway.
So I honestly don't know what to do! I'll have to think about this and let the initial excitement calm down it seems.
You mentioned that you've not yet had to report/contribute anything to Procyon since there's been no need, however you don't yet get compilable source-code when using Enigma (or at least that's implied). Even if MC was still 100% obfuscated on decompilation, it should still be compilable *if* the decompiler is solid - so doesn't this suggest that there's issues with Procyon that still need to be solved?
Unless there's something I'm missing. I have no idea how MCP do it, there's no such thing as a perfect java decompiler that I know of - yet MCP seem to have done it, so I don't know the details there.
EDIT : Also, it should not allow to rename a class to an existing class, but that shouldn't be hard to fix.
I think that's outside the scope of Enigma, such things are best left to precompiler's and/or IDE's. Enigma has much more important work ahead of it than preventing idiotic human error
Wow, this makes reverse engineering so much easier, thank you.
But unfortunately, in v0.4.2 beta, renaming functions just doesn't work.
EDIT : Also, it should not allow to rename a class to an existing class, but that shouldn't be hard to fix.
These problems are already fixed in the latest commits. I'll put out a new release (0.5 beta) after I fix a few more bugs on my list.
You mentioned that you've not yet had to report/contribute anything to Procyon since there's been no need, however you don't yet get compilable source-code when using Enigma (or at least that's implied). Even if MC was still 100% obfuscated on decompilation, it should still be compilable *if* the decompiler is solid - so doesn't this suggest that there's issues with Procyon that still need to be solved?
Unless there's something I'm missing. I have no idea how MCP do it, there's no such thing as a perfect java decompiler that I know of - yet MCP seem to have done it, so I don't know the details there.
I've reported a couple issues to Procyon, but I haven't contributed any PRs yet. Procyon does a great job of giving you source code (that actually compiles) when you give it good bytecode. The problem is, I'm feeding Procyon bad bytecode because the obfuscator throws away a ton of metadata. Half of the job of Enigma is picking names for all the classes/fields/methods/etc. The other half is to fix up the bytecode before giving it to Procyon so Procyon can figure out what the hell is going on. The trick here is the JVM and the Java language spec are two different things. Bytecode that runs on the JVM doesn't necessarily have to resemble the java language spec. Think about Groovy, scala, etc. They all compile to bytecode and run on the JVM, but they're not necessarily java. The obfuscators really take advantage of this in making the bytecode hard to reverse engineer.
I think that's outside the scope of Enigma, such things are best left to precompiler's and/or IDE's. Enigma has much more important work ahead of it than preventing idiotic human error
Nah, people make mistakes. There should either be a way to undo those mistakes, or a mechanism to prevent them from happening in the first place. The latter is usually much easier to implement. =)
I've reported a couple issues to Procyon, but I haven't contributed any PRs yet. Procyon does a great job of giving you source code (that actually compiles) when you give it good bytecode. The problem is, I'm feeding Procyon bad bytecode because the obfuscator throws away a ton of metadata. Half of the job of Enigma is picking names for all the classes/fields/methods/etc. The other half is to fix up the bytecode before giving it to Procyon so Procyon can figure out what the hell is going on. The trick here is the JVM and the Java language spec are two different things. Bytecode that runs on the JVM doesn't necessarily have to resemble the java language spec. Think about Groovy, scala, etc. They all compile to bytecode and run on the JVM, but they're not necessarily java. The obfuscators really take advantage of this in making the bytecode hard to reverse engineer.
Hmm. What do you mean by "meta data" exactly? Is it like... a heuristic reconstruction of code based on debugging symbols? If it's complicated/lengthy to explain, nevermind
Ah, so Enigma is *also* a bytecode parser or "pre-processor" rather than just feeding something into Procyon. Well that's awfully fancy. Sadly I know next to nothing about Java bytecode; Dalvik is a completely different beast to Java (it's a little funny how the one language can produce two entirely, architecturally-different executables) so I can't help with that though I wish I could.
And with that said, yeah I know what you mean about JVM vs Java, or rather "the java language" vs "a java executable"
Nah, people make mistakes. There should either be a way to undo those mistakes, or a mechanism to prevent them from happening in the first place. The latter is usually much easier to implement. =)
Hmm. What do you mean by "meta data" exactly? Is it like... a heuristic reconstruction of code based on debugging symbols? If it's complicated/lengthy to explain, nevermind
Ah, so Enigma is *also* a bytecode parser or "pre-processor" rather than just feeding something into Procyon. Well that's awfully fancy. Sadly I know next to nothing about Java bytecode; Dalvik is a completely different beast to Java (it's a little funny how the one language can produce two entirely, architecturally-different executables) so I can't help with that though I wish I could.
"metadata" includes things like method argument names, inner class relationships, generic types, labeling of fields/methods created by the java compiler to implement language features like enumerations and covariant return types. All of this information is removed by the obfuscator, but you need it to decopmile bytecode into compilable source code.
Enigma tries to infer the missing information from the bytecode, and from user input, so we can get as close to the original source code as possible. Think of Enigma as a bytecode pre-processor and decompiler frontend. =)
"metadata" includes things like method argument names, inner class relationships, generic types, labeling of fields/methods created by the java compiler to implement language features like enumerations and covariant return types. All of this information is removed by the obfuscator, but you need it to decopmile bytecode into compilable source code.
Enigma tries to infer the missing information from the bytecode, and from user input, so we can get as close to the original source code as possible. Think of Enigma as a bytecode pre-processor and decompiler frontend. =)
Ah right. Yet another difference between JVM and Dalvik then, we don't get any of this fancy metadata to work with - just hack around with bytecode converted to plain text Regardless, sounds very complex - I salute you sir.
I've converted the 1.8-pre3 mappings to the official 1.8 release version. =)
There's also a new version of Enigma, v0.5 beta. This version fixes a lot of bugs and adds a couple new features. The biggest new feature is the mappings conversion tool, but it's not exposed in the GUI at all. I just run it from Eclipse directly. Other new features include:
grouping classes by package in the classes selector.
added checks to prevent naming classes/fields/methods/arguments the same name as other classes/fields/methods/arguments
field/method/constructor reference browser now shows public/protected/private access
added crash reporter so crashes are no longer hidden on the console
Whoah! Both Bukkit and Spigot downloads have been taken down due to a DMCA take down request by one of the main bukkit devs Wesley Wolfe (Wolvereness).
It's a legitimate question. It's not obvious to tell what's going on from just looking at Enigma's GUI. Right now, Enigma lacks the ability to create a working source tree for Minecraft. I'm actually working on implementing that now. The eventual goal is to be able to pick a Minecraft jar, pick a mappings file, click a button, and export a fully-compilable source tree for Minecraft.
#enigma is not available, though it is on Esper. Not that it really matters.
Edit: Registered #enigma and #enigma-mappings on Esper if we want those.
Enigma v0.4 beta is out.Main new features:support for interfaces: "Show Interfaces" command in editor. Method rename now renames all methods related by interfaces as well as related by inheritance.
source export function: write all deobfuscated source files to a folder. The exported source doesn't compile yet. I'm working on that...
EDIT: v0.4.1 beta is out! Fixed bug with sort order when writing mappings.
v0.4.2 beta is out now that fixes a problem with method signatures.
Last I used Procyon though, it had a lot of problems. Granted I was using it to analyze reverse-compiled Dalvik executables, but the conversion from dex <> jar is a lossless process and the AOSP does indeed use javac. Maybe Androids' use of Java is a bit obscure to traditional Java and Procyon just isn't mature enough to support it yet; but jd-gui was most often more accurate. How far as Procyon come along? Do you contribute to it?
I have a vague question, if you don't mind. Does MCP completely decompile and deobfuscate Minecraft into working code? Judging from what you see when making mods, this seems to be the case - but I seriously find this hard to believe. What's the story there, is that true (and is that what's required for your decompiler/reverse-compiler to get to)?
EDIT: Nevermind, I see this:
Christ, you're crazy. Crazy brilliant xD
I'll hve to try Enigma out with my real-job reverse engineering tasks. If it turns out well, I may very well contribute via monetary means. And I don't mean have-a-beer-on-me donations; reverse engineering is hard work and good money
Procyon has been amazing to work with. The project seems very mature now and it emits good source code. I've run into a few technical issues with Procyon, but since it's open source, I can see what's going on. Also, Procyon's maintainer has been super helpful and is just generally pleasant to work with. I would contribute to Procyon, but it hasn't been necessary yet.
I tried jd before I found Procyon, but it was just garbage. js is too dated to be of any use for this work.
MCP completely decompiles and de-obfuscates Minecraft jars in to a working source tree. It does a pretty good job of it too, so it's a tough competitor to beat. =)
And yes, reverse engineering is pretty challenging, but I think it's also a lot of fun. Enigma/M3L is a hobby project for me. It's a nice easy break from my actual job.
Feel free to use Enigma at work if you actually find it useful there. I never thought someone outside the Minecraft community would find it useful though, so that's kind of a nice surprise. Contributions are always welcome too. I have a bitcoin address, or there's always Patreon.
Wait, what?!
Yes, that's right. We have a working tool now that will (mostly) automate the conversion of mappings between different versions of Minecraft. Expect that Enigma will be able to keep up with Minecraft snapshots now. Imagine being able to test out your mods on new Minecraft snapshots... =)
Yeah well, reverse engineering in a job role is not a very common thing (and it's not the only thing I do, other roles are minor Java development, main role is embedded Linux/kernel work). I work with a lot of Chinese developers who, for whatever reason, don't agree to sharing source code but are fine with us reverse engineering to customize instead (makes no sense but whatever). Enigma would have a lot of popularity among the Android hobbyist/hacking scene such as XDA for example. As you said, jd produces garbage - there are other decompilers I've tried with varying degrees of success (nested classes and synchronized clauses will often fail to decompile in anything I've used), and of course that lack of Java > 5 support as the Procyon page says.
Very nice! I'm super interested in helping out with development of Enigma/M3L, but I really don't know if I'll have the time to invest anything worthwhile plus I am still quite new to Minecraft modding and by no means an expert on Forge modding even. But even the fact that Enigma has deobfuscation that I can actually (and easily) fix makes it far more worthwhile to use than Forge, however we're still a quite a distance from getting a full blown API/loader anyway.
So I honestly don't know what to do! I'll have to think about this and let the initial excitement calm down it seems.
You mentioned that you've not yet had to report/contribute anything to Procyon since there's been no need, however you don't yet get compilable source-code when using Enigma (or at least that's implied). Even if MC was still 100% obfuscated on decompilation, it should still be compilable *if* the decompiler is solid - so doesn't this suggest that there's issues with Procyon that still need to be solved?
Unless there's something I'm missing. I have no idea how MCP do it, there's no such thing as a perfect java decompiler that I know of - yet MCP seem to have done it, so I don't know the details there.
I think that's outside the scope of Enigma, such things are best left to precompiler's and/or IDE's. Enigma has much more important work ahead of it than preventing idiotic human error
These problems are already fixed in the latest commits. I'll put out a new release (0.5 beta) after I fix a few more bugs on my list.
I've reported a couple issues to Procyon, but I haven't contributed any PRs yet. Procyon does a great job of giving you source code (that actually compiles) when you give it good bytecode. The problem is, I'm feeding Procyon bad bytecode because the obfuscator throws away a ton of metadata. Half of the job of Enigma is picking names for all the classes/fields/methods/etc. The other half is to fix up the bytecode before giving it to Procyon so Procyon can figure out what the hell is going on. The trick here is the JVM and the Java language spec are two different things. Bytecode that runs on the JVM doesn't necessarily have to resemble the java language spec. Think about Groovy, scala, etc. They all compile to bytecode and run on the JVM, but they're not necessarily java. The obfuscators really take advantage of this in making the bytecode hard to reverse engineer.
Nah, people make mistakes. There should either be a way to undo those mistakes, or a mechanism to prevent them from happening in the first place. The latter is usually much easier to implement. =)
Hmm. What do you mean by "meta data" exactly? Is it like... a heuristic reconstruction of code based on debugging symbols? If it's complicated/lengthy to explain, nevermind
Ah, so Enigma is *also* a bytecode parser or "pre-processor" rather than just feeding something into Procyon. Well that's awfully fancy. Sadly I know next to nothing about Java bytecode; Dalvik is a completely different beast to Java (it's a little funny how the one language can produce two entirely, architecturally-different executables) so I can't help with that though I wish I could.
And with that said, yeah I know what you mean about JVM vs Java, or rather "the java language" vs "a java executable"
Ah nice fair enough, if it's easy then great
"metadata" includes things like method argument names, inner class relationships, generic types, labeling of fields/methods created by the java compiler to implement language features like enumerations and covariant return types. All of this information is removed by the obfuscator, but you need it to decopmile bytecode into compilable source code.
Enigma tries to infer the missing information from the bytecode, and from user input, so we can get as close to the original source code as possible. Think of Enigma as a bytecode pre-processor and decompiler frontend. =)
Ah right. Yet another difference between JVM and Dalvik then, we don't get any of this fancy metadata to work with - just hack around with bytecode converted to plain text Regardless, sounds very complex - I salute you sir.
There's also a new version of Enigma, v0.5 beta. This version fixes a lot of bugs and adds a couple new features. The biggest new feature is the mappings conversion tool, but it's not exposed in the GUI at all. I just run it from Eclipse directly. Other new features include:
- Local thread about this -
- bukkit thread -
- Spigot thread -
Other than Vanilla + Realms we might be stuck on 1.6.4 or 1.7.2 / 1.7.10 servers for a while...
EDIT: Here is one persons explanation of what is happening: DMCA Claim
- The Cubic Chunks Mod is back! Be a part of it's rebirth and Development.
-- Robinton's Mods: [ Mirror ] for some of his Mods incl Cubic Chunks Mod, due to DropBox broken links.
- Dungeon Generator for the Open Cubic Chunks Mod
- QuickSAVE-QuickLOAD for the Open Cubic Chunks Mod
THE MICRO$OFT APOCALYPSE IS ON!!!
32 bit redstone computer with 64*64 pixel 16 color gpu:
http://www.minecraftforum.net/forums/minecraft-discussion/redstone-discussion-and/2116999-computer-command-block-computer-now-running-with
(giant banner removed out of modesty).