Modding Discussion Modding Guides and General Modding Discussion--REDUX

Discussion in 'Mods' started by cantorsdust, Mar 10, 2016.

Thread Status:
Not open for further replies.
  1. cantorsdust

    cantorsdust Existential Complex

    ARE YOU A BAD ENOUGH DUDE TO JUMP ON #stardewvalley-mods ?

    (you are, come join us)

    Old thread here



    MODS


    GO HERE FOR MOD RELEASE FORUMS
    GO HERE FOR THE CHUCKLEFISH OFFICIAL STARDEW VALLEY MOD REPOSITORY

    With the addition of a Mod Releases section of the site, I won't be maintaining a mod list anymore. I may add a recommended mods section later, as the modding community matures.



    DOWNLOADS

    APIs


    Stardew Modding API (SMAPI) originally by Zoryn4163, now updated and maintained by ClxS and Zoryn.

    Storm API by Demmonic et al. Compilable source code on Github available for modders, general public BETA binaries are automatically created regularly from the source are available.


    Mod Managers

    SDVMM by yuukiw. Simple mod manager with a GUI that can install and update Storm and SMAPI for you as well as installing and managing DLL and XNB mods.


    Tools

    XNB Node (0.2.1) by Draivin. RECOMMENDED FOR XNB PACKING, UNPACKING (now packs/unpacks PNG, text (YAML) data), and map files. PNG transparency issues should be fixed.

    XNB Extract by Drogean. RECOMMENDED FOR XNB PACKING, UNPACKING. A simple toolkit put together to make unpacking using XNB Node as easy as possible. Includes XNB Node in the download.

    tIDE. RECOMMENDED A map editor for unpacked map files.

    Notepad++ RECOMMENDED(editing text files)

    XNB Builder NOT RECOMMENDED, FOR POSTERITY Alternative for PNGs if you're having trouble with white borders or transparency issues with your sprites.

    XNB Parser 0.7 NOT RECOMMENDED, FOR POSTERITY. (unpacks PNG data, unpacks text as XML, but the XML file cannot be repacked!)

    SimpleXNBDemapper (1.1) by Pinkishu / Inari-Whitebear. Thanks to her work with draivin, the program's functionality has been added to XNB Node 0.2.1. NOT RECOMMENDED, FOR POSTERITY. Map / level editor.

    Chicklet Save Editor by stefanhaan.
    Save Game Editor by etsmsj.

    Stardew Rocks by nictuku. Uploads your save file to a website, which produces a screenshot of your farm from the save file. Very neat.


    Guides

    SMAPI Mod and Configuration Tutorials by the SMAPI team

    SMAPI Modder's Guide by OrSpeeder

    Storm Video Modding Guide by kodfod

    XNB Modder's Guide by Alpha_Omegasis

    Beginner's Guide to XNB Node and Graphics Editing by cure

    Step by Step Guide with Pictures for Unpacking and Repacking Mods by cantorsdust

    Audio Editing Guide by Drogean

    Map Editing Video How to by Pinkishu / Inari-Whitebear. See also the instructions on the github.

    Font Editing Guide by Chrisblue

    Crops Modding Guide by SgtPineapple1st.

    Pixel Editors Guide by Kamirose


    Modding Streams

    SgtPineapple1st (forums thread)



    MODDING PROGRESS REPORT


    The original thread was meant to be a brief progress report on my work unpacking and editing XNB files, then grew much larger. I will continue the tradition of the progress report here.

    Modding currently includes 3 basic categories:
    1. XNB modding
    2. API modding
    3. Audio modding

    XNB Modding

    XNB modding was the first method to be discovered.

    Stardew Valley has a simple file structure. All game data is found within the Stardew Valley\Content folder. Unfortunately, all data (except for sounds) is stored as .xnb (XNB ) files, which are not the easiest to mod. XNB files are binary containers for Microsoft's XNA game development platform. XNB files are binary objects which essentially contain versions of ordinary game data such as pictures, text files, etc. Editing them in straightforward programs like Notepad++ etc is not possible.

    To edit XNB files, one must first unpack them, like you would a ZIP. There are several programs available under the Tools section above. Currently, draivin's XNB Node is a very versatile and functional tool and the preferred method for all XNB file unpacking and repacking.

    All graphical data in the game can be extracted as .png (PNG) files within XNB files within the folders Content and the subfolders Content\Animals, Buildings, Characters, LooseSprites, Maps, mines, Minigames, Portraits, TerrainFeatures, and TileSheets.

    All fonts in the game are extracted as a PNG file paired with a text (as of v0.1.0 of XNB Node, a YAML, but previously JSON and XML) file within XNB files within the subfolder Content\Fonts. The text file appears to note which pixels in the PNG correspond to which symbol.

    All game data / tweaking values can be extracted as text within XNB files within the subfolder Content\Data. The data within are stored as text strings of unlabeled values, making modding these values trial and error without the guides above or knowledge of the source code.

    Game map levels can now be extracted with XNB Node as well. These may be edited with tIDE, a map editor.

    Files can then be edited and repacked using XNB Node.

    XNB files can then replace the existing game files and thus replace their data.


    Progress

    XNB modding has enabled a large range of changes to the game. These include:
    1. Graphic replacer mods by editing PNGs
    2. Game mechanic tweak mods by editing Content\Data text files
    3. Map level edits by editing Content\Maps map files with tIDE
    4. One very impressive item addition mod adding rice by SgtPineapple1st

    Limitations

    XNB modding has many limitations, and it is not an ideal modding solution. XNB modding
    1. Is mostly limited to replacing existing graphics and tweaking existing game parameters
    2. Makes direct edits to base game files
    3. Does not allow multiple mods to the same file
    4. Must be updated every time a patch updates the file

    Future Goals
    1. Continued improvement on our knowledge of game parameter editing.
    Frankly, there isn't much room for improvement within the limitations of XNB modding. All XNB files can now be fully unpacked and repacked thanks to improvements to XNB node.

    I hope that with proper modding API support, these limitations can be circumvented and XNB modding can become obsolete.


    API Modding

    APIs are application programming interfaces. They sit between a modder's program and the game and allow the modder to access and edit the game. A full featured modding API will allow modders with programming skills to make edits to the game's values in real time, execute their own code, and add and extend game features and content. You may be familiar with modding APIs in other games, most notably Forge for Minecraft.

    A good API has several key features:
    1. It functions as a layer between the mods and the game and protects the game from unsafe edits
    2. Because it sits between the mods in the game, it can update for each new patch, preventing mods from breaking with each game update
    3. It makes modding easier by adding new functions that mods can use
    There are currently 2 APIs for Stardew Valley.


    Stardew Modding API (SMAPI)

    The first to be developed was Stardew Modding API (SMAPI) by Zoryn and continued by ClxS. SMAPI had a very minimal protective layer and allowed mods to access and run game code directly. This made modding very straightforward, but also prone to breaking. SMAPI functions using an event system. It listens for certain events to happen in the game--a new day starting, the location changing, etc--and allows mods to "subscribe" to those events. When those events fired, mods could execute their code.

    Several mods have been produced for SMAPI altering many game functions. Adding new content to the game, such as art or new items, has not yet been done except for trivial proofs of concept.

    Zoryn left SMAPI to join the work on Storm, as he felt SMAPI to be unmaintainable in the long term. ClxS has continued development of SMAPI.

    More recently, Zoryn has returned to the SMAPI project, leaving Storm. He plans to update the API along with ClxS until a more fully-featured alternative is developed.


    Storm

    The second API to be developed was Storm, started by Demmonic.

    Storm offers a much more robust protective layer between the mods and the base game. This makes modding more stable, but also initially more difficult. Accessors, wrappers, and proxies must be created to allow mods safe access to the base game. This means that modders may lack functions available in SMAPI until they are added in by the Storm team. The long term goal of the Storm Team is to make all base game methods and fields available to the modder through the API.

    Storm also features an event system, but it is more extensive. Modders can subscribe to Pre and Post events that fire before and after the functions are called, allowing them finer control of the game code.

    Storm has also produced proofs of concept for adding items, but no non-trivial item addition mod has yet been made.

    Storm development has slowed recently. Its founder, Demmonic, and one of its main developers, mdbell, have left the project.


    Progress

    Significant progress with API modding has been made, including:
    1. Major edits to game functions
    2. Addition of new game features
    3. Trivial creation of new content
    4. Addition of new areas via API loading alone.

    Limitations

    A number of limitations to API modding remain.
    1. API modding has not yet provided methods for non-trivial creation of new items.
    2. The Event system for both APIs does not yet provide events for all methods
    3. SMAPI allows unfiltered mod access to base game code.
    4. Storm does not yet offer access to every method and field.

    Future Goals
    1. Development of a fully featured API with access to base game methods as well as a robust event system.
    2. Development of standard code libraries for things like mod configuration, in game menu creation, etc.
    3. True content addition--creation of new items, NPCs, dialogue, crops, tools etc

    With the slowing of Storm's development, there is no public fully-featured API. SMAPI remains the most popular in terms of mod variety and ease of use. The "holy grail" would be an API that combine's SMAPI's ease of use and access to every method and field and Storm's versatile event system and access layer.


    Audio Modding

    Sounds are stored within Content\XACT under Wave Bank.xwb (XWB ). This is the only content not stored in XNB files.

    Drogean has taken the lead on audio modding and has produced tools for unpacking and repacking the audio files as well as a guide. His audio mods, to my knowledge, remain the only audio mods, but are very technically impressive.

    Progress, Limitations, and Future Goals?



    MODDING HOW TOS AND FAQS


    ERROR REPORTING

    I wanted to add this because a clear, understandable, and courteous error report is a great boon to any modder. If you can write error reports as good as this, you will get help every time.

    Look at how good this report is. Look at what it does:
    1. It uses proper grammar so I can understand what the hell you're trying to say.
    2. It explains exactly what the problem is.
    3. It outlines, step by step, the process that caused the error.
    4. It gives me the command used.
    5. It shows me the full error, including any logs if generated.
    6. It lists attempts at a fix and alternate programs used.
    7. It is consistently courteous and non-accusatory.
    Write an error report like this, and you will get helped.



    XNB MODDING

    The basic steps in this process are:
    1. Install XNB Node.
    2. Unpack XNB files.
    3. Edit XNB files
    4. Pack XNB files
    5. Replace Stardew Valley's base game XNB files with your newly edited ones.
    6. Load your game and test your changes.
    I have made a step by step picture guide for this process. It was written for an earlier version of XNB Node, but the process is exactly the same.

    Be sure to make backups of the original base game files during this process. If you are missing files, you can use Steam - > Stardew Valley (right click) - > Properties - > Local Files - > Verify Game cache to replace them. This will replace any missing OR EDITED files you have, so back up your mods.

    There is an excellent general guide to XNB modding by Alpha_Omegasis
    which covers the basics of packing and unpacking, graphics file editing, and text file editing.


    1. Unpack XNB files.

    To unpack graphics and game data:
    This is done with an XNB unpacker. Use Draivin's XNB Node as posted earlier in this thread. Mad props to Draivin! Unzip the file so that xnb_node.cmd is in the same directory as your Stardew Valley.exe. Again, use the utility with the command line. Start cmd.exe, cd over to the Stardew Valley folder.

    Unpack files by using "xnb_node.cmd extract [input directory] [output directory].

    Do note that xnb_node creates an associated YAML file for each PNG file it extracts. These YAML files are associated with the PNG and should be kept with it. It will need the YAML with the PNG when you repack the file. THE PACKER WILL NOT WORK WITHOUT THE YAML WITH THE PNG.


    Now that you have extracted files, it is time to edit them.


    2. Edit your files.
    Open up ContentUnpacked\Minigames\TitleButtons.png in your graphic editor of choice. Scribble across the Stardew Valley title. Congratulations, you have edited the file. Be sure to save your work under the same name, TitleButtons.png

    For YAML, many good text editors exist. I recommend Notepad++.


    3. Repack your loose, edited files into .xnb files.

    To pack PNG and YAML files

    Pack files by using "xnb_node.cmd pack [input directory or file] [output directory] from the command line. To pack PNGs, make sure the associated YAML is with them in the same directory. THE PACKER WILL NOT WORK WITHOUT THE YAML WITH THE PNG.

    Take your new XNB, make sure it is the same name of the file you are trying to mod, and replace it in the original file's location in Stardew Valley\Content. It would be wise to keep a backup of the original file, but I'm a post, not a cop.


    4. Start your game and confirm your changes.


    Example XNB Mod

    Below is a sample file of TitleButtons.xnb which should show the word MOD written in black over the title on Stardew Valley's start screen. It is a basic proof of concept. Place it in Stardew Valley\Content\Minigames. Be sure to back up the original!

    http://www.mediafire.com/download/x5gr39cf4l22d4w/TitleButtons.xnb



    API MODDING

    API modding requires basic programming skills and knowledge of or ability to learn C#.

    Installing Visual Studio 2015 Community Edition is free. It is the reference IDE for Storm, and works great for C# development, including git integration.

    The basic process for creating an API mod for Stardew Valley is:
    1. Make sure you have installed Microsoft .NET Framework 4.5.2, 4.0, and XNA 4.0. You likely have already if you're running Stardew Valley, but double check.
    2. Install Visual Studio 2015 Community Edition (recommended) or another C# IDE of your choice
    3. If using VS 2015, install XNA Game Studio 4.0 for VS 2015.
    4. (Optional but recommended) Install your version control software of choice. Storm and SMAPI both use Git.
    5. Install Storm or SMAPI. Note for Storm users--the publicly available binaries do not offer the developer's console. Download and compile Storm's source to get a binary with the console.
    6. Create a new C# Class Library (DLL) project in your IDE
    7. Add either StormLoader.exe (Storm) or StardewModdingAPI.exe and StardewValley.exe as references to the project.
    8. You may need to add additional references--usually Microsoft.XNA.Framework, Microsoft.XNA.Framework.Game.
    9. Code your project. Refer to the example mods for both APIs for a guide as to how to set it up. Generally, the structure is to set up events you are subscribing to, and then executing code when those events fire.
    10. Compile your project.
    11. Install your mod and test it.
    12. (Optional but recommended) Curse frequently when it doesn't work.
    Tutorials for creating a SMAPI mod and adding a mod config are available on the SMAPI Github Wiki.

    Most modders make regular reference to decompiled source code for Stardew Valley. Decompilers I have found to be best for Stardew Valley have been dotPeek and ILSPY, both free.

    Please feel free to join the IRC channel #stardewvalley-mods on @freenode.net for modding questions. Most of the Storm team hangs out there.

    You may find when modding with Storm that there is an event, method, or field you would like. Asking nicely on IRC is most likely to be successful. Alternatively, you can open an issue on the Storm GitLab.

    I don't have a picture guide for API Modding (yet), although, frankly, if you cannot complete steps 1-7 above, you're probably going to have a hard time making a mod.


    AUDIO MODDING

    I will, again, just refer to Drogean's guide, as I know little about audio modding myself.
    Audio Editing Guide by Drogean



    CURRENT HANGUPS

    1. Cannot repack XML files. Gives a BUILD FAILED error. The logfile reports "XML is not in the XNA intermediate format. Missing XnaContent root element." Googling for the problem shows that when XML files are turned into XNB files, they require an XNA Content root element that specifies the type of the data being saved. See here. In short, XML files require an <XnaContent> tag at the beginning, followed by a <Asset Type="whatever the type is"> tag. The XML files in Stardew Valley's XNB files lack these tags. Further reading noted that this tag is part of an automatic creation process for XML to XNB creation added later on in the XNA timeline. Earlier projects had to create a custom content pipeline to convert XML to XNB files. I suspect Stardew Valley was started early enough to require such a pipeline. If so, it may be difficult to ever create XNB files from XML files without developer support.

    SOLVED THANKS TO DRAIVIN'S XNB NODE!

    2. Cannot unpack special file types within XNB files. These appear to be file types not handled by the XNB Parser program I posted above. As the program must write a handler for each specific file type, someone would have to manually write one for each currently unsupported file type. I lack the programming ability to do that, but someone out there should be able to. This currently prevents modding any XNB containing those file types. Even if the other files are output successfully from such an XNB, they would not be able to be repacked within a new functional XNB while the other files that we cannot yet unpack are missing.

    SOLVED--XNB Node 0.2.1 supports unpacking all game files.

    3. Editing existing game functions and adding new features. Becoming more and more possible through APIs, but only for those with C# programming experience.

    IN PROGRESS--APIs in development.

    4. Item Creation. The XNB data format is limiting--only one person can make edits at a time to the same file. API modding would allow for more flexibility, adding items on the fly from multiple people.

    IN PROGRESS--APIs in development.

    5. Creating new NPCs. Again theoretically possible but impractical with XNB modding. Would require creation of a portrait, sprite, schedule, dialogue, and insertion into existing locations, events, and festivals.

    THEORETICALLY POSSIBLE, CURRENTLY UNATTEMPTED

    6. Modifying existing NPC dialogue, making other NPCs marriagable. More research required as to the requirements for marriage and whether anything is hardcoded as a requirement. Presumably API modding will make this practically easier.

    MORE RESEARCH REQUIRED

    7. Modifying existing maps. Success by Pinkishu into editing and adding onto existing maps. Adding to the right and down would be most practical for now--there are coordinates that count from the upper left corner of the screen required for moving between areas, NPC positioning, and likely other things.

    XNB MODS COMPLETED. No API development yet.

    8. Adding new maps. Level creation and packing is now possible, but loading and integrating into the game is not. There are many hardcoded scripts for each area, however, and it's not clear how functional new areas would be yet.

    Demonstrated successfully by Advize in Farm Expansion

    9. Translation projects. Most text is located in Content\Characters\Dialogue, Content\Characters\schedules, Content\Data\Events, Content\Data\Festivals, and Content\Data\TV. Item descriptions are scattered through the files of Content\Data. There is some text that is hardcoded by the game. The biggest barrier to translation projects, however, are the font files. They are only in English characters, and no accented or unusual characters included. So any translation not using Latin characters is not possible until fonts are created. I font XNB files can now be packed and unpacked by XNB Node 0.1.0, so creation of a font is now theoretically possible.

    The second barrier would be loading new font graphics into the game. Font characters are defined using an associated text file with a list of glyphs corresponding to each letter. You would have to edit the font graphic, then define each character in a list of glyphs. This would then have to be repacked and successfully loaded by the game. To date this has not been achieved, but it is not known whether it is not currently possible.

    BEGUN! Apparently it's not too bad to add fonts, unlike items. See Chrisblue's post.


    CONCLUSION

    In conclusion, I'd like to thank ConcernedApe for a fantastic game. Let me know what you guys would like to see in the OP, guides you want created, etc. I'm very much open to feedback.
     
      Last edited: Mar 25, 2016
      Mju, embeys, ajikumara and 17 others like this.
    • Drogean

      Drogean Scruffy Nerf-Herder

      this really needs to get stickied

      BUMP
       
        Jaata, nickelcurry and Ragnar6645 like this.
      • Acrimonia

        Acrimonia Subatomic Cosmonaut

        Step 11 is a satisfying fuel for the flames of discontent, but I'm more of a fan of optional step 11.b; take a sip of tea or coffee, followed by a deep breath. Try, try again. :nod:
        Seconding that this would make a great sticky.
         
          Ragnar6645 likes this.
        • Ragnar6645

          Ragnar6645 Seal Broken

          Nice post cantorsdust, very well written.

          And yeah this needs to be stickied.
           
            Acrimonia likes this.
          • stringstorm

            stringstorm Astral Cartographer

            What exactly am I supposed to type into the command prompt if I want to extract the .png from the .xnb? I've tried what the guide told me to do and it literally just broke my game. Had to download again. :(
             
            • cantorsdust

              cantorsdust Existential Complex

              xnb_node.cmd extract [your XNB file name] [the same file name but ending in .png, not .xnb]

              Note that this command requires the relative paths from xnb_node.cmd to the file if the file is not in the same directory as xnb_node.cmd.


              edit: Instructions from the guide itself:

              The extract command for xnb_node.cmd is this:

              "xnb_node.cmd extract [input folder] [output folder]"

              OR "xnb_node.cmd extract [input file with file extension] [output file WITH FILE EXTENSION]"

              Eg to extract your entire Stardew Valley\Content folder, type "xnb_node.cmd extract Content ContentUnpacked"

              To extract Content\Characters\Abigail.xnb, type "xnb_node.cmd extract Content\Characters\Abigail.xnb Abigail.png" Note the file extension there for Abigail.png.


              There's nothing in the extract command that would break your game. It does not delete your content. Your content stays right where it is.
               
              • stringstorm

                stringstorm Astral Cartographer

                Managed to do it on one of the portraits but can't open it on gimp for some reason.
                 
                • Trestkon

                  Trestkon Void-Bound Voyager

                  This thread needs to get stickied, very informative, luckily I bookmarked this earlier so I didn't lose it.
                   
                  • mafi_93

                    mafi_93 Subatomic Cosmonaut

                    Hiii I have problems to repack my fonts. Simply XmlNode does nothing. Attached my png and yaml.
                    Can someone tell me what is wrong?
                     

                      Attached Files:

                    • LinkOfStardewValley

                      LinkOfStardewValley Space Spelunker

                      Hi! I've been trying to mod Stardew Valley so I could change the image of one of the hats in Stardew Valley to look like Link's hat from Zelda.

                      I've followed your guide on imgur, but when I get to Step 5 and I enter in xnb_node.cmd Content ContentUnpacked (I already have both of those folders in my Stardew Valley file), it says this:

                      "The system cannot find the path specified"

                      I've not made any typos or anything, so I don't know what's up.


                      EDIT: I forgot to move the "app" folder over to the Stardew Valley folder. Silly me. It works now!
                       

                        Attached Files:

                        Last edited: Mar 10, 2016
                      • cantorsdust

                        cantorsdust Existential Complex

                        I assume you mean XNB Node. If you're referring to SimpleXNBDemapper, that's really just for map files, and XNB Node now has its functionality. I acked them for you. In the future, refer to the picture guide, specifically steps 8 and 11, for you how repack loose files.
                         

                          Attached Files:

                        • Drogean

                          Drogean Scruffy Nerf-Herder

                          updated the sound guide with new tools for v1.05
                           
                          • yuukiw

                            yuukiw Scruffy Nerf-Herder

                            @cantorsdust could you exchange /add this sdvmm link with that one just to be safe. The bat version on Dropbox still works fine for smapi mods but won't be update anymore.

                            Link :https://github.com/yuukiw/StardewValleyMM/releases/tag/1.2a

                            @ topic nice new post. I would have an basic mod idea so I will look into moding once I have my Physik exam behind me.
                             
                            • keyten

                              keyten Cosmic Narwhal

                              HI! Don't you now what is the point of files FarmerSounds.xgs and Sound bank.xwb?
                               
                              • Drogean

                                Drogean Scruffy Nerf-Herder

                                sorry i've actually never extracted them
                                 
                                • Acrimonia

                                  Acrimonia Subatomic Cosmonaut

                                  99% sure that FarmerSounds is a blank 'dummy file' but that you could (theoretically?) add info and calls for sound, perhaps damage 'grunts' or the like?
                                  As to Sound bank.xwb...

                                  It is a mystery.
                                  Probably a few unsorted sounds, the audio equivilent of LooseSprites?

                                  Edit: Also, I'm gonna be revising everything lots when the weekend finally hits, but I'd love any feedback regarding this protoguide.
                                  Any ideas on how to make it prettier and better are welcome, though I might end up just re-writing it from the ground up. Hopefully it'll help some prospective modders, though!
                                  and if I could figure out how to get npcs in the game proper that'd be nice too
                                   
                                    Last edited: Mar 11, 2016
                                  • keyten

                                    keyten Cosmic Narwhal

                                    Well well well. Looks like xgs file contains names of all 355 sounds in the end. IF we can modify this file we will probably be able to add our own music and sounds!
                                     
                                      Acrimonia and Drogean like this.
                                    • Acrimonia

                                      Acrimonia Subatomic Cosmonaut

                                      Woah...
                                      That would be a huge boon, much better then having to overrwrite sounds..!
                                      ... I would've never expected that from the name, though. Huh.
                                       
                                      • cantorsdust

                                        cantorsdust Existential Complex

                                        Updated.
                                         
                                        • mafi_93

                                          mafi_93 Subatomic Cosmonaut

                                          Yes, I meant XMLNode. I did the steps well (and I have other files repacked before). Not if it's because PNG and yaml not have the same name. You simply put it in testinput and put "xnb_node.cmd pack ..." and it worked?
                                           
                                          Thread Status:
                                          Not open for further replies.

                                          Share This Page