[golang/go] cmd/go: opt-in for //go:embed to *not* ignore files and empty dirs (#43854)
DRANK

This issue follows from [#42328](https://github.com/golang/go/issues/42328#issuecomment-765245498) which is now closed, and where @mpx suggested I create a new one. # Overview I have a //go:embed of ~700 files, ~70 directories and where some files have a leading underscore. These are not dot files, nor go source files, and it took me some time to work out that the reason I couldn't open some of them without error was that they were excluded from the build. The [draft proposal doesn't mention this restriction](https://go.googlesource.com/proposal/+/master/design/draft-embed.md). It was written in July and the exclusion of these files was made in December by a change resulting from #42328. It was counterintuitive to me that files would be arbitrarily excluded because they're considered "hidden" on some platforms. Why counterintuitive? I don't know of any other tools that automatically exclude dot or leading-underscore files, other than ls for dot files, which provides the well-known `-a` escape hatch. Git, find, tar, etc., include such files by default. Even the macOS Finder allows you to toggle display dot files with `⌘-shift-.`. Related to this, I sometimes use "filesystem templates" that include empty directories that are later populated by code: these are either initially extracted to the filesystem, or used with a virtual in-memory filesystem. To embed such a template using //go:embed, I'd have to add placeholder files like a `readme.txt` with contents "This file intentionally left blank.". (When forced to do this, it is preferable to use `.empty` dot files, but they're also unsupported by //go:embed!) I've also experienced a similar *chicken or the egg* problem with an empty embed dir; see [1] below. Sometimes we need to include *everything* in an embed of a directory tree. Please give us a way out! :) # Suggestions and feedback A simple opt-in flag like `//go:embed -all dir` to include everything in `dir` without filtering, and without declaring an error if the directory is empty [1]. This flag would instruct //go:embed to include all files, including those having a leading underscore, and any dot files. For the edge-case of an embed of a file called "-all", and for which you could use a "--" terminator: `//go:embed -all -- -all`. The `flag` package [makes this implementation straightforward](https://github.com/golang/go/issues/42328#issuecomment-765586787). Otherwise having some explicit filtering option in the directive (with an "otherwise include-all") would be welcomed, as was suggested in #42325. The proposal in #42328 for //go:embed dir vs. //go:embed dir/** is very subtle and reminds me of the arcane build constraint syntax that was thankfully recently revisited. An explicit inclusion-list, written by hand or code generated, is a big turn-off. [1] In one project, I have a package that embeds a directory tree that is generated by another program. The generator needs to use the same package, which leads to a *chicken or the egg* problem. I currently resolve this by having a Makefile add a `readme.txt` file into the embed directory before each build (which unfortunately remains in the embed). I am not using the Makefile for any other purpose here. # Workarounds 1. Write more code and wrap my file tree in a store-only ZIP file and //go:embed that. 1. Use a 3rd party embed code-generator tool and eat the slower build times. 1. Using some encoding to escape filenames that //go:embed otherwise denies me from using naturally. 1. Add placeholder (non-dot) files in empty directories that are managed by Makefiles. The first two options are not preferred because they burden the project with an external generator. The third option would force design changes to a larger surface area of an otherwise already complex project just to simplify the build. I'd love to avoid the use of Makefiles in the last workaround, especially when they serve no other purpose.

github.com
Related Topics: Go (programming language)
1 comments