在windows上用mingw64编译emacs的feature/native-comp分支
gccemacs(http://akrl.sdf.org/gccemacs.html)已经合并到emacs的开发分支 feature/native-comp 。
gccemacs使用gcc 5.0后新增的jit功能,把elisp文件编译成一个个动态链接库。这样比emacs解析执行lisp代码要快一倍以上。
gccemacs在linux上开发的,但是没有porting到windows上,趁着春节有空,尝试着在mingw64中编译最新的gcc和emacs的native-comp分支。
由于gccjit是在linux上开发的,只考虑了对linux的支持,并没有考虑对windows支持,需要修改代码才能在windows上运行。gcc的libgccjit在mingw64上编译有点复杂,emacs的native-comp分支也是。
编译支持libgccjit的gcc
在最新的mingw64里,gcc的版本是v9.2.0,从 gcc -v
看,mingw64的gcc版本没有把libgccjit编译进去,所以不支持libgccjit。
在pacman中找不到gcc的libgccjit的package。
# 没有gccjit相关package pacman -Ss jit
用源代码编译gcc的trunk的最新代码。
# depth=1 只clone最新的,gcc太多了,网速慢,clone太慢,我们也不需要完整的版本。 git clone --depth 1 git://gcc.gnu.org/git/gcc.git
编译
重点关注一下几个编译选项:
只要支持c和c++就可以了。lto(link time optimize)看文档是默认的。jit看gccjit的文档上有,不知道不加有没有问题。
--enable-languages=c,jit,lto,c++
下面这2个参数是编译gccjit需要的。
--enable-host-shared \ --enable-languages=jit \
如果enable的话,configure的时候会比较慢。
--disable-bootstrap
下面是完整的编译选项:
cd /e/workspace/gcc/build ../gcc/configure --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --with-arch=x86-64 --with-tune=generic --enable-languages=c,jit,lto,c++ --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-filesystem-ts=yes --enable-libstdcxx-time=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-isl-version-check --enable-lto --enable-libgomp --disable-multilib --enable-checking=release --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --enable-plugin --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev2, Built by MSYS2 project. Albert 2020.01.20' --with-gnu-as --with-gnu-ld \ --enable-host-shared \ --enable-languages=jit \ --disable-bootstrap
编译路径
gcc编译时,建议新建一个目录,在新建目录中进行编译。不要直接在gcc的目录下执行 configure 。在gcc的同级目录下建 build 目录,然后再configure。
Albert@Albert MINGW64 /e/workspace/gcc $ ls -tlr total 20 drwxr-xr-x 1 Albert None 0 Jan 20 17:37 gcc # gcc的代码 drwxr-xr-x 1 Albert None 0 Jan 21 16:25 build # 编译目录
pic/libiberty.a 报错
make[3]: *** No rule to make target '../build-x86_64-w64-mingw32/libiberty/pic/libiberty.a', needed by 'build/genmddeps.exe'. Stop.
enable-host-shared后,link的时候找pic目录下的libiberty.a,但是找不到。
--enable-host-shared \ --enable-languages=jit \
解决方法
在libiberty目录下 mkdir pic && cp -p libiberty.a pic
,手工copy一份,测试是ok的。
报错 - 无法编译xgcc.exe
可能makefile里面的依赖关系不对,需要手工make一下
cd build && make xgcc.exe
代码修改
在 jit-tempdir.c 中增加mkdtemp()函数
参考 https://doxygen.postgresql.org/mkdtemp_8c_source.html
git里面的代码修改 fchmod\dlopen\dlerror
https://gcc.gnu.org/ml/jit/2015-q3/msg00126.html
参考https://stackoverflow.com/questions/6431345/how-to-specify-dll-onload-function-for-mingw32 把jit-playback.c里面的
- 用LoadLibrary()替换dlopen()
- 用SetLastError(0)和GetLastError()替换dlerror()
- 安装dlfcn
编译gccjit的时候不知道安装dlfcn后还要不要改共享库相关的代码。
在编译emacs native-comp的时候需要安装
dlfcn
。# 在数据库中search dlfcn包 pacman -Ss dlfcn # 安装dlfcn pacman -S dlfcn $ find /mingw64/ -name 'libdl*' -ls 3659174697621175 20 -rwxr-xr-x 1 Albert None 20470 6月 21 2019 /mingw64/bin/libdl.dll 2533274790778565 8 -rw-r--r-- 1 Albert None 4586 6月 21 2019 /mingw64/lib/libdl.a 2251799814067910 4 -rw-r--r-- 1 Albert None 3068 6月 21 2019 /mingw64/lib/libdl.dll.a
注释掉fchmod()
mingw64中没有函数定义导致编译gcc报错。
windows不像linux,无所谓文件权限,不需要生成的共享库是700的权限。直接注释掉。
新增 gcc_jit_context_new_rvalue_from_long_long_int()
具体修改
Albert@Albert MINGW64 /e/workspace/gcc/gcc $ git diff diff --git a/gcc/jit/jit-playback.c b/gcc/jit/jit-playback.c index da687002a..0757c08bc 100644 --- a/gcc/jit/jit-playback.c +++ b/gcc/jit/jit-playback.c @@ -42,6 +42,9 @@ along with GCC; see the file COPYING3. If not see #include <pthread.h> +/* mingw64 */ +#include <windows.h> + #include "jit-playback.h" #include "jit-result.h" #include "jit-builtins.h" @@ -613,6 +616,30 @@ new_rvalue_from_const <long> (type *type, } } +/* Specialization of making an rvalue from a const, for host <long long int>. */ + +template <> +rvalue * +context:: +new_rvalue_from_const <long long int> (type *type, + long long int value) +{ + // FIXME: type-checking, or coercion? + tree inner_type = type->as_tree (); + if (INTEGRAL_TYPE_P (inner_type)) + { + tree inner = build_int_cst (inner_type, value); + return new rvalue (this, inner); + } + else + { + REAL_VALUE_TYPE real_value; + real_from_integer (&real_value, VOIDmode, value, SIGNED); + tree inner = build_real (inner_type, real_value); + return new rvalue (this, inner); + } +} + /* Specialization of making an rvalue from a const, for host <double>. */ template <> @@ -1818,7 +1845,7 @@ block (function *func, /* Compile a playback::context: - - Use the context's options to cconstruct command-line options, and + - Use the context's options to construct command-line options, and call into the rest of GCC (toplev::main). - Assuming it succeeds, we have a .s file. - We then run the "postprocess" vfunc: @@ -2155,15 +2182,15 @@ playback::compile_to_file::copy_file (const char *src_path, gcc_assert (total_sz_in == total_sz_out); if (get_logger ()) - get_logger ()->log ("total bytes copied: %ld", total_sz_out); + get_logger ()->log ("total bytes copied: %lld", total_sz_out); /* Set the permissions of the copy to those of the original file, in particular the "executable" bits. */ - if (fchmod (fileno (f_out), stat_buf.st_mode) == -1) - add_error (NULL, - "error setting mode of %s: %s", - dst_path, - xstrerror (errno)); + /* if (fchmod (fileno (f_out), stat_buf.st_mode) == -1) */ + /* add_error (NULL, */ + /* "error setting mode of %s: %s", */ + /* dst_path, */ + /* xstrerror (errno)); */ fclose (f_out); } @@ -2641,16 +2668,20 @@ dlopen_built_dso () JIT_LOG_SCOPE (get_logger ()); auto_timevar load_timevar (get_timer (), TV_LOAD); void *handle = NULL; - const char *error = NULL; + /* const char *error = NULL; */ + DWORD error = 0; result *result_obj = NULL; /* Clear any existing error. */ - dlerror (); - - handle = dlopen (m_tempdir->get_path_so_file (), - RTLD_NOW | RTLD_LOCAL); - if ((error = dlerror()) != NULL) { - add_error (NULL, "%s", error); + /* dlerror (); */ + SetLastError(0); + + /* handle = dlopen (m_tempdir->get_path_so_file (), */ + /* RTLD_NOW | RTLD_LOCAL); */ + handle = LoadLibrary(m_tempdir->get_path_so_file ()); + /* if ((error = dlerror()) != NULL) { */ + if ((error = GetLastError()) != 0) { + add_error (NULL, "%ld", error); } if (handle) { diff --git a/gcc/jit/jit-recording.c b/gcc/jit/jit-recording.c index b73cd76a0..acb40730e 100644 --- a/gcc/jit/jit-recording.c +++ b/gcc/jit/jit-recording.c @@ -4480,6 +4480,7 @@ recording::global::write_reproducer (reproducer &r) /* Explicit specialization of the various mementos we're interested in. */ template class recording::memento_of_new_rvalue_from_const <int>; template class recording::memento_of_new_rvalue_from_const <long>; +template class recording::memento_of_new_rvalue_from_const <long long int>; template class recording::memento_of_new_rvalue_from_const <double>; template class recording::memento_of_new_rvalue_from_const <void *>; @@ -4617,6 +4618,69 @@ recording::memento_of_new_rvalue_from_const <long>::write_reproducer (reproducer m_value); } +/* The make_debug_string specialization for <long long int>, rendering it as + (TARGET_TYPE)LITERAL + e.g. + "(long long int)42". */ + +template <> +string * +memento_of_new_rvalue_from_const <long long int>::make_debug_string () +{ + return string::from_printf (m_ctxt, + "(%s)%lli", + m_type->get_debug_string (), + m_value); +} + +/* The get_wide_int specialization for <long long int>. */ + +template <> +bool +memento_of_new_rvalue_from_const <long long int>::get_wide_int (wide_int *out) const +{ + *out = wi::shwi (m_value, sizeof (m_value) * 8); + return true; +} + +/* The write_reproducer specialization for <long long int>. */ + +template <> +void +recording::memento_of_new_rvalue_from_const <long long int>::write_reproducer (reproducer &r) +{ + const char *id = r.make_identifier (this, "rvalue"); + + /* We have to special-case LONG_MIN, since e.g. + -9223372036854775808L + is parsed as + -(9223372036854775808L) + and hence we'd get: + error: integer constant is so large that it is unsigned [-Werror] + Workaround this by writing (LONG_MIN + 1) - 1. */ + if (m_value == LONG_LONG_MIN) + { + r.write (" gcc_jit_rvalue *%s =\n" + " gcc_jit_context_new_rvalue_from_long_long_int (%s, /* gcc_jit_context *ctxt */\n" + " %s, /* gcc_jit_type *numeric_type */\n" + " %lldL - 1); /* long long int value */\n", + id, + r.get_identifier (get_context ()), + r.get_identifier_as_type (m_type), + m_value + 1); + return; + } + + r.write (" gcc_jit_rvalue *%s =\n" + " gcc_jit_context_new_rvalue_from_long_long_int (%s, /* gcc_jit_context *ctxt */\n" + " %s, /* gcc_jit_type *numeric_type */\n" + " %lldL); /* long long int value */\n", + id, + r.get_identifier (get_context ()), + r.get_identifier_as_type (m_type), + m_value); + } + /* The make_debug_string specialization for <double>, rendering it as (TARGET_TYPE)LITERAL e.g. diff --git a/gcc/jit/jit-result.c b/gcc/jit/jit-result.c index c10e5a13c..69c80f191 100644 --- a/gcc/jit/jit-result.c +++ b/gcc/jit/jit-result.c @@ -22,6 +22,9 @@ along with GCC; see the file COPYING3. If not see #include "system.h" #include "coretypes.h" +/* mingw64 */ +#include <windows.h> + #include "jit-common.h" #include "jit-logging.h" #include "jit-result.h" @@ -49,7 +52,8 @@ result::~result() { JIT_LOG_SCOPE (get_logger ()); - dlclose (m_dso_handle); + /* dlclose (m_dso_handle); */ + FreeLibrary((HMODULE)m_dso_handle); /* Responsibility for cleaning up the tempdir (including "fake.so" within the filesystem) might have been handed to us by the playback::context, @@ -72,15 +76,18 @@ get_code (const char *funcname) JIT_LOG_SCOPE (get_logger ()); void *code; - const char *error; + /* const char *error; */ + DWORD error; /* Clear any existing error. */ - dlerror (); - - code = dlsym (m_dso_handle, funcname); - - if ((error = dlerror()) != NULL) { - fprintf(stderr, "%s\n", error); + /* dlerror (); */ + SetLastError(0); + + /* code = dlsym (m_dso_handle, funcname); */ + code = (void *)GetProcAddress((HMODULE)m_dso_handle, funcname); + /* if ((error = dlerror()) != NULL) { */ + if ((error = GetLastError()) != 0) { + fprintf(stderr, "%ld\n", error); } return code; @@ -99,15 +106,19 @@ get_global (const char *name) JIT_LOG_SCOPE (get_logger ()); void *global; - const char *error; + /* const char *error; */ + DWORD error; /* Clear any existing error. */ - dlerror (); + /* dlerror (); */ + SetLastError(0); - global = dlsym (m_dso_handle, name); + /* global = dlsym (m_dso_handle, name); */ + global = (void *)GetProcAddress((HMODULE)m_dso_handle, name); - if ((error = dlerror()) != NULL) { - fprintf(stderr, "%s\n", error); + /* if ((error = dlerror()) != NULL) { */ + if ((error = GetLastError()) != 0) { + fprintf(stderr, "%ld\n", error); } return global; diff --git a/gcc/jit/jit-tempdir.c b/gcc/jit/jit-tempdir.c index 10c528faf..457591708 100644 --- a/gcc/jit/jit-tempdir.c +++ b/gcc/jit/jit-tempdir.c @@ -24,6 +24,316 @@ along with GCC; see the file COPYING3. If not see #include "jit-tempdir.h" +/*------------------------------------------------------------------------- + * + * mkdtemp.c + * create a mode-0700 temporary directory + * + * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group + * + * + * IDENTIFICATION + * src/port/mkdtemp.c + * + * This code was taken from NetBSD to provide an implementation for platforms + * that lack it. (Among compatibly-licensed implementations, the OpenBSD + * version better resists denial-of-service attacks. However, it has a + * cryptographic dependency.) The NetBSD copyright terms follow. + *------------------------------------------------------------------------- + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <stddef.h> +#include <stdarg.h> +#include <sys/types.h> +#include <errno.h> +#include <locale.h> + + /* + * Supplement to <sys/stat.h>. + * + * We must pull in sys/stat.h before this part, else our overrides lose. + */ +#define lstat(path, sb) stat(path, sb) + +#define _DIAGASSERT(x) do {} while (0) + + + /* $NetBSD: gettemp.c,v 1.17 2014/01/21 19:09:48 seanb Exp $ */ + + /* + * Copyright (c) 1987, 1993 + * The Regents of the University of California. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + + #if HAVE_NBTOOL_CONFIG_H + #include "nbtool_config.h" + #endif + + #if !HAVE_NBTOOL_CONFIG_H || !HAVE_MKSTEMP || !HAVE_MKDTEMP + + #ifdef NOT_POSTGRESQL + #include <sys/cdefs.h> + #if defined(LIBC_SCCS) && !defined(lint) + #if 0 + static char sccsid[] = "@(#)mktemp.c 8.1 (Berkeley) 6/4/93"; + #else + __RCSID("$NetBSD: gettemp.c,v 1.17 2014/01/21 19:09:48 seanb Exp $"); + #endif + #endif /* LIBC_SCCS and not lint */ + #endif + + #include <sys/types.h> + #include <sys/stat.h> + + #include <assert.h> + // #include <ctype.h> + #include <errno.h> + #include <fcntl.h> + #include <stdio.h> + #include <stdlib.h> + #include <unistd.h> + + #ifdef NOT_POSTGRESQL + #if HAVE_NBTOOL_CONFIG_H + #define GETTEMP __nbcompat_gettemp + #else + #include "reentrant.h" + #include "local.h" + #define GETTEMP __gettemp + #endif + #endif + + static int + GETTEMP(char *path, int *doopen, int domkdir) + { + char *start, + *trv; + struct stat sbuf; + int pid; + + /* + * To guarantee multiple calls generate unique names even if the file is + * not created. 676 different possibilities with 7 or more X's, 26 with 6 + * or less. + */ + // static char xtra[2] = "aa"; + static char xtra[2] = "a"; + int xcnt = 0; + + _DIAGASSERT(path != NULL); + /* doopen may be NULL */ + + pid = getpid(); + + /* Move to end of path and count trailing X's. */ + for (trv = path; *trv; ++trv) + if (*trv == 'X') + xcnt++; + else + xcnt = 0; + + /* Use at least one from xtra. Use 2 if more than 6 X's. */ + if (xcnt > 0) + { + *--trv = xtra[0]; + xcnt--; + } + if (xcnt > 5) + { + *--trv = xtra[1]; + xcnt--; + } + + /* Set remaining X's to pid digits with 0's to the left. */ + for (; xcnt > 0; xcnt--) + { + *--trv = (pid % 10) + '0'; + pid /= 10; + } + + /* update xtra for next call. */ + if (xtra[0] != 'z') + xtra[0]++; + else + { + xtra[0] = 'a'; + if (xtra[1] != 'z') + xtra[1]++; + else + xtra[1] = 'a'; + } + + /* + * check the target directory; if you have six X's and it doesn't exist + * this runs for a *very* long time. + */ + for (start = trv + 1;; --trv) + { + if (trv <= path) + break; + if (*trv == '/') + { + int e; + + *trv = '\0'; + e = stat(path, &sbuf); + *trv = '/'; + if (e == -1) + return doopen == NULL && !domkdir; + if (!S_ISDIR(sbuf.st_mode)) + { + errno = ENOTDIR; + return doopen == NULL && !domkdir; + } + break; + } + } + + for (;;) + { + if (doopen) + { + if ((*doopen = + open(path, O_CREAT | O_EXCL | O_RDWR, 0600)) >= 0) + return 1; + if (errno != EEXIST) + return 0; + } + else if (domkdir) + { + if (mkdir(path, 0700) >= 0) + // if (mkdir(path) >= 0) + return 1; + if (errno != EEXIST) + return 0; + } + else if (lstat(path, &sbuf)) + return errno == ENOENT ? 1 : 0; + + /* tricky little algorithm for backward compatibility */ + for (trv = start;;) + { + if (!*trv) + return 0; + if (*trv == 'z') + *trv++ = 'a'; + else + { + // 不能用isdigit,要用宏定义的 + if (ISDIGIT((unsigned char) *trv)) + *trv = 'a'; + else + ++*trv; + break; + } + } + } + /* NOTREACHED */ + } + + #endif /* !HAVE_NBTOOL_CONFIG_H || !HAVE_MKSTEMP || + * !HAVE_MKDTEMP */ + + + /* $NetBSD: mkdtemp.c,v 1.11 2012/03/15 18:22:30 christos Exp $ */ + + /* + * Copyright (c) 1987, 1993 + * The Regents of the University of California. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + + #if HAVE_NBTOOL_CONFIG_H + #include "nbtool_config.h" + #endif + + #if !HAVE_NBTOOL_CONFIG_H || !HAVE_MKDTEMP + + #ifdef NOT_POSTGRESQL + + #include <sys/cdefs.h> + #if defined(LIBC_SCCS) && !defined(lint) + #if 0 + static char sccsid[] = "@(#)mktemp.c 8.1 (Berkeley) 6/4/93"; + #else + __RCSID("$NetBSD: mkdtemp.c,v 1.11 2012/03/15 18:22:30 christos Exp $"); + #endif + #endif /* LIBC_SCCS and not lint */ + + #if HAVE_NBTOOL_CONFIG_H + #define GETTEMP __nbcompat_gettemp + #else + #include <assert.h> + #include <errno.h> + #include <stdio.h> + #include <stdlib.h> + #include <unistd.h> + #include "reentrant.h" + #include "local.h" + #define GETTEMP __gettemp + #endif + + #endif + + char * + mkdtemp(char *path) + { + _DIAGASSERT(path != NULL); + + return GETTEMP(path, NULL, 1) ? path : NULL; + } + + #endif /* !HAVE_NBTOOL_CONFIG_H || !HAVE_MKDTEMP */ /* Construct a tempdir path template suitable for use by mkdtemp e.g. "/tmp/libgccjit-XXXXXX", but respecting the rules in @@ -51,7 +361,7 @@ make_tempdir_path_template () tmpdir_len = strlen (tmpdir_buf); /* tmpdir_buf should now have a dir separator as the final byte. */ gcc_assert (tmpdir_len > 0); - gcc_assert (tmpdir_buf[tmpdir_len - 1] == DIR_SEPARATOR); + /* gcc_assert (tmpdir_buf[tmpdir_len - 1] == DIR_SEPARATOR); */ file_template_buf = "libgccjit-XXXXXX"; file_template_len = strlen (file_template_buf); @@ -101,9 +411,9 @@ gcc::jit::tempdir::create () return false; log ("m_path_tempdir: %s", m_path_tempdir); - m_path_c_file = concat (m_path_tempdir, "/fake.c", NULL); - m_path_s_file = concat (m_path_tempdir, "/fake.s", NULL); - m_path_so_file = concat (m_path_tempdir, "/fake.so", NULL); + m_path_c_file = concat (m_path_tempdir, "\\fake.c", NULL); + m_path_s_file = concat (m_path_tempdir, "\\fake.s", NULL); + m_path_so_file = concat (m_path_tempdir, "\\fake.so", NULL); /* Success. */ return true; diff --git a/gcc/jit/jit-tempdir.h b/gcc/jit/jit-tempdir.h index 7bbf9ea2f..39542df5e 100644 --- a/gcc/jit/jit-tempdir.h +++ b/gcc/jit/jit-tempdir.h @@ -88,4 +88,5 @@ class tempdir : public log_user } // namespace gcc +char * mkdtemp(char *); #endif /* JIT_TEMPDIR_H */ diff --git a/gcc/jit/libgccjit++.h b/gcc/jit/libgccjit++.h index 82a62d614..2208f78c6 100644 --- a/gcc/jit/libgccjit++.h +++ b/gcc/jit/libgccjit++.h @@ -878,6 +878,16 @@ context::new_rvalue (type numeric_type, value)); } +inline rvalue +context::new_rvalue (type numeric_type, + long long int value) const +{ + return rvalue ( + gcc_jit_context_new_rvalue_from_long_long_int (m_inner_ctxt, + numeric_type.get_inner_type (), + value)); +} + inline rvalue context::zero (type numeric_type) const { diff --git a/gcc/jit/libgccjit.c b/gcc/jit/libgccjit.c index 83055fc29..2f3076cf5 100644 --- a/gcc/jit/libgccjit.c +++ b/gcc/jit/libgccjit.c @@ -1197,6 +1197,19 @@ gcc_jit_context_new_rvalue_from_long (gcc_jit_context *ctxt, ->new_rvalue_from_const <long> (numeric_type, value)); } +gcc_jit_rvalue * +gcc_jit_context_new_rvalue_from_long_long_int (gcc_jit_context *ctxt, + gcc_jit_type *numeric_type, + long long int value) +{ + RETURN_NULL_IF_FAIL (ctxt, NULL, NULL, "NULL context"); + JIT_LOG_FUNC (ctxt->get_logger ()); + RETURN_NULL_IF_FAIL_NONNULL_NUMERIC_TYPE (ctxt, numeric_type); + + return ((gcc_jit_rvalue *)ctxt + ->new_rvalue_from_const <long long int> (numeric_type, value)); +} + /* Public entrypoint. See description in libgccjit.h. This is essentially equivalent to: diff --git a/gcc/jit/libgccjit.h b/gcc/jit/libgccjit.h index 21a0dc09b..21237c31a 100644 --- a/gcc/jit/libgccjit.h +++ b/gcc/jit/libgccjit.h @@ -812,6 +812,11 @@ gcc_jit_context_new_rvalue_from_long (gcc_jit_context *ctxt, gcc_jit_type *numeric_type, long value); +extern gcc_jit_rvalue * +gcc_jit_context_new_rvalue_from_long_long_int (gcc_jit_context *ctxt, + gcc_jit_type *numeric_type, + long long int value); + extern gcc_jit_rvalue * gcc_jit_context_zero (gcc_jit_context *ctxt, gcc_jit_type *numeric_type); diff --git a/gcc/jit/libgccjit.map b/gcc/jit/libgccjit.map index 4514bd3aa..488a3a089 100644 --- a/gcc/jit/libgccjit.map +++ b/gcc/jit/libgccjit.map @@ -62,6 +62,7 @@ LIBGCCJIT_ABI_0 gcc_jit_context_new_rvalue_from_double; gcc_jit_context_new_rvalue_from_int; gcc_jit_context_new_rvalue_from_long; + gcc_jit_context_new_rvalue_from_long_long_int; gcc_jit_context_new_rvalue_from_ptr; gcc_jit_context_new_string_literal; gcc_jit_context_new_struct_type;
测试libgccjit是否正常
用libgccjit文档中的helloworld.c (https://gcc.gnu.org/onlinedocs/jit/intro/tutorial01.html) 。
在main函数打开jit的log,打印stdout中,可以更好地确认libgccjit编译是否正常。
int main() { ... /* 把gccjit的log打印到stdout上,用这个就可以了。 */ gcc_jit_context_set_logfile (ctxt, stdout, 0, 0); /* Populate the context. */ create_code (ctxt); /* 生成汇编语言是ok的,在e:/tmp目录下 */ /* gcc_jit_context_compile_to_file(ctxt, GCC_JIT_OUTPUT_KIND_ASSEMBLER, "/tmp/hello.s"); */ /* 生成dll是ok的,在e:/tmp目录下 */ /* gcc_jit_context_compile_to_file(ctxt, GCC_JIT_OUTPUT_KIND_DYNAMIC_LIBRARY, "/tmp/hello.dll"); */ /* [2020-01-21 周二 22:09:45] 执行报 Null result */ /* gcc_jit_context_dump_reproducer_to_file(ctxt, "/tmp/hello-generated.c"); */ ... }
编译emacs的feature/native-comp branch
https://git.savannah.gnu.org/cgit/emacs.git/log/?h=feature/native-comp
修改src/comp.c的代码
- mingw64中不支持signal SIGIO ,直接注释掉。
- mingw64中configure时居然不支持 setjmp 。在setjmp.h中有 _setjmp ,里面的宏太多了,搞不清楚是用哪个,编译时报错,提示有2个参数。从setjmp.h里看 _setjmp(jmp_buf, void * ptr),第二个参数一般是0。反正没看懂。先填个0进去了。
$ git diff diff --git a/src/comp.c b/src/comp.c index 290fc3a9c4..43b5273d45 100644 --- a/src/comp.c +++ b/src/comp.c @@ -69,7 +71,7 @@ #define DECL_BLOCK(name, func) \ #ifdef HAVE__SETJMP #define SETJMP _setjmp #else -#define SETJMP setjmp +#define SETJMP _setjmp #endif #define SETJMP_NAME SETJMP @@ -1268,10 +1270,12 @@ emit_limple_push_handler (gcc_jit_rvalue *handler, gcc_jit_rvalue *handler_type, NULL, comp.handler_jmp_field), NULL); - + /* mingw64 _setjmp need 2 args */ + args[1] = gcc_jit_context_null(comp.ctxt, comp.void_ptr_type); gcc_jit_rvalue *res; res = - emit_call (intern_c_string (STR (SETJMP_NAME)), comp.int_type, 1, args, false); + emit_call (intern_c_string (STR (SETJMP_NAME)), comp.int_type, 2, args, false); emit_cond_jump (res, handler_bb, guarded_bb); } @@ -1838,7 +1842,9 @@ #define ADD_IMPORTED(f_name, ret_type, nargs, args) \ ADD_IMPORTED (push_handler, comp.handler_ptr_type, 2, args); args[0] = gcc_jit_type_get_pointer (gcc_jit_struct_as_type (comp.jmp_buf_s)); - ADD_IMPORTED (SETJMP_NAME, comp.int_type, 1, args); + args[1] = comp.void_ptr_type; + ADD_IMPORTED (SETJMP_NAME, comp.int_type, 2, args); ADD_IMPORTED (record_unwind_protect_excursion, comp.void_type, 0, NULL); @@ -3148,7 +3154,7 @@ DEFUN ("comp--compile-ctxt-to-file", Fcomp__compile_ctxt_to_file, sigemptyset (&blocked); sigaddset (&blocked, SIGALRM); sigaddset (&blocked, SIGINT); - sigaddset (&blocked, SIGIO); + /* sigaddset (&blocked, SIGIO); */ pthread_sigmask (SIG_BLOCK, &blocked, &oldset); } emit_ctxt_code ();
编译
改为代码后,如果make时总是报错,最好 make clean
再重新configure。emacs的configure过程太慢了,10分钟都完不了。make时,编译src目录下的可执行文件,几分钟就能完成,在elisp文件编译eln时,会非常慢,可能要1个小时。
configure的参数参考 http://chriszheng.science/2015/03/19/Chinese-version-of-Emacs-building-guideline/ 。
./autogen.sh PKG_CONFIG_PATH=/mingw64/lib/pkgconfig ./configure --host=x86_64-w64-mingw32 \ --with-wide-int \ --without-dbus \ --with-jpeg --with-xpm --with-png --with-tiff --with-rsvg --with-xml2 \ --with-gnutls=ifavailable \ --without-compress-install -C 'CFLAGS=-g -static -g3' \ --without-imagemagick --with-nativecomp # 6个进程编译会快不少。 make -j6 NATIVE_FAST_BOOT=1 # 安装目录 e:/emacs make install prefix=/e/emacs
- make时报 error 127 ,可能是 byte-run.el、subr.el 等eln编译有问题,导致emacs在生成pdump文件有问题,启动报错,可以rm *.eln,再试试。
- 有时候native compile某个elisp文件会coredump,只能再执行make。
native comp的有时候会coredump,可能和_setjmp有关,通过gdb没看出来问题在哪里。
在异常处理时可能会segment fault。
如何知道load了几个eln文件 在linux上可以在/proc中看 /proc/pid/maps 文件。在windows上,使用 vmmap (https://docs.microsoft.com/zh-cn/sysinternals/downloads/vmmap)。
grep eln /proc/`pidof emacs`/maps|awk '{print $6}'|sort -u
执行
修改mingw64的环境变量,把编译后的gcc加进去。新增 LIBRARY_PATH ,貌似 LD_LIBRARY_PATH 加了也没用。
export PATH=$PATH:/e/workspace/gcc/build/gcc:/mingw64/lib/gcc/x86_64-w64-mingw32/10.0.1:/mingw64/x86_64-w64-mingw32/lib export LIBRARY_PATH=/e/workspace/gcc/build/gcc:/mingw64/x86_64-w64-mingw32/lib:/mingw64/bin
debug
按gdb的提示,把emacs的.gdbinit 加到 ~/.gdbinit中。debug时可以看见elisp执行的堆栈。gdb时emacs会比较慢。
可以参考 etc/DEBUG 的内容。
cd src gdb emacs run # 或者 run -Q 不加载init.el # segfault后 bt
功能测试
cd /e/workspace/emacs_src/emacs28
./emacs/src/emacs -batch -l ert -l ./emacs/test/src/comp-tests.el -f ert-run-tests-batch-and-exit
fail了2个test case。
2 unexpected results: FAILED comp-tests-bootstrap FAILED comp-tests-fixnum
comp-tests-bootstrap
搞不懂为什么自己编译自己生成的c文件为什么变量名字不一样,导致生成的eln文件也不一样。
为什么linux上就没问题。难道mingw64上的gcc和linux上的有什么地方不一样?
comp-tests-fixnum
由于mingw64中的long和int是4字节,long long int是8字节。在linux上long是8字节。comp.c中的代码默认long是8字节。导致类型转换时精度降低,无法通过测试。
libgccjit.c中只有gcc_jit_context_new_rvalue_from_long() ,没有 gcc_jit_context_new_rvalue_from_long_long_int() ,自己加一个。然后修改comp.c。
直接修改gcc_jit_context_new_rvalue_from_long() ,直接cast为 long long int ,也是一个方法。
make[1]: 进入目录“/e/workspace/emacs_src/emacs28/emacs/src” GEN globals.h CC comp.o In file included from comp.c:30: comp.c: In function 'Fcomp__init_ctxt': lisp.h:1134:30: warning: overflow in conversion from 'long long int' to 'long int' changes value from '2305843009213693951' to '-1' [-Woverflow] 1134 | #define MOST_POSITIVE_FIXNUM (EMACS_INT_MAX >> INTTYPEBITS) | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ comp.c:3077:15: note: in expansion of macro 'MOST_POSITIVE_FIXNUM' 3077 | MOST_POSITIVE_FIXNUM); | ^~~~~~~~~~~~~~~~~~~~ lisp.h:1135:30: warning: overflow in conversion from 'long long int' to 'long int' changes value from '-2305843009213693952' to '0' [-Woverflow] 1135 | #define MOST_NEGATIVE_FIXNUM (-1 - MOST_POSITIVE_FIXNUM) | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ comp.c:3087:15: note: in expansion of macro 'MOST_NEGATIVE_FIXNUM' 3087 | MOST_NEGATIVE_FIXNUM); | ^~~~~~~~~~~~~~~~~~~~
Benchmark
在win10上测试,cpu是intel i5-8520U。
第一次测试
# 未加 -batch -Q 参数,在gui中,可能对测试性能有影响。 ./emacs -l E:/workspace/elisp-benchmarks/elisp-benchmark.el --eval '(elb-run)'
Summary results: bubble.el byte time: 48.196685s native time: 22.676739s boost 112.537989% bubble-no-cons.el byte time: 140.633307s native time: 38.527450s boost 265.021062% fibn.el byte time: 137.998609s native time: 74.574372s boost 85.048302% fibn-rec.el byte time: 111.793463s native time: 38.222614s boost 192.479899% fibn-tc.el byte time: 105.274320s native time: 27.331516s boost 285.175561% inclist-tc.el byte time: 114.527039s native time: 2.016713s boost 5578.896253% listlen-tc.el byte time: 102.875721s native time: 0.843463s boost 12096.826772% inclist-no-type-hints.el byte time: 149.213448s native time: 6.573894s boost 2169.787861% inclist-type-hints.el byte time: 145.497633s native time: 4.850708s boost 2899.513329% nbody.el byte time: 196.386510s native time: 138.361917s boost 41.936824% dhrystone.el byte time: 612.382567s native time: 499.131995s boost 22.689504% Total byte time: 667.153574s native time: 287.974774s boost 131.670839%
第二次测试
./emacs -batch -l E:/workspace/elisp-benchmarks/elisp-benchmark.el --eval '(elb-run)' -Q # centos 7.2 on virtualbox, intel i5-8250u 3c/3t 3G RAM. export LD_LIBRARY_PATH=/usr/local/lib ~/emacs_src/emacs/src/emacs -batch -l ~/elisp/elisp-benchmarks/elisp-benchmark.el --eval '(elb-run)' -Q
bubble.el byte time: 100.621556s native time: 20.373809s boost 393.876997% bubble-no-cons.el byte time: 136.691828s native time: 35.343340s boost 286.754132% fibn.el byte time: 121.392839s native time: 67.133157s boost 80.823969% fibn-rec.el byte time: 109.009100s native time: 38.573689s boost 182.599624% fibn-tc.el byte time: 103.859580s native time: 24.524760s boost 323.488670% inclist-tc.el byte time: 101.627444s native time: 2.023227s boost 4923.037158% listlen-tc.el byte time: 92.898451s native time: 0.697208s boost 13224.352417% inclist-no-type-hints.el byte time: 124.360178s native time: 5.929948s boost 1997.154612% inclist-type-hints.el byte time: 131.778451s native time: 5.045148s boost 2511.983851% nbody.el byte time: 493.410192s native time: 122.395166s boost 303.128823% dhrystone.el byte time: 349.269993s native time: 216.528584s boost 61.304335% Total byte time: 552.128101s native time: 161.879026s boost 241.074514% [2020-02-02 周日 14:06:59] bubble.el byte time: 138.523664s native time: 21.739262s boost 537.204998% bubble-no-cons.el byte time: 185.583256s native time: 41.305705s boost 349.292068% fibn.el byte time: 191.460652s native time: 74.148040s boost 158.214043% fibn-rec.el byte time: 120.859300s native time: 36.182891s boost 234.023337% fibn-tc.el byte time: 107.256165s native time: 24.652318s boost 335.075375% inclist-tc.el byte time: 111.663299s native time: 1.793739s boost 6125.169827% listlen-tc.el byte time: 102.768693s native time: 0.708594s boost 14403.184193% inclist-no-type-hints.el byte time: 174.505982s native time: 5.239305s boost 3230.708596% inclist-type-hints.el byte time: 191.516524s native time: 5.280125s boost 3527.121025% nbody.el byte time: 525.441776s native time: 142.299708s boost 269.250073% dhrystone.el byte time: 442.338356s native time: 261.127992s boost 69.395227% Total byte time: 624.094766s native time: 173.716878s boost 259.259718% [2020-02-07 周五 17:22:21] bubble.el byte time: 102.715487s native time: 21.517346s boost 377.361320% bubble-no-cons.el byte time: 107.479181s native time: 33.150175s boost 224.219046% fibn.el byte time: 125.995228s native time: 67.917504s boost 85.512159% fibn-rec.el byte time: 105.926435s native time: 38.019204s boost 178.612974% fibn-tc.el byte time: 100.486451s native time: 25.953107s boost 287.184667% inclist-tc.el byte time: 109.109979s native time: 2.030358s boost 5273.928095% listlen-tc.el byte time: 113.611750s native time: 0.827511s boost 13629.334112% inclist-no-type-hints.el byte time: 147.052826s native time: 6.744831s boost 2080.229957% inclist-type-hints.el byte time: 136.069896s native time: 4.647000s boost 2828.123434% nbody.el byte time: 501.310644s native time: 128.587244s boost 289.860322% dhrystone.el byte time: 368.023414s native time: 216.367226s boost 70.092033% Total byte time: 596.685092s native time: 170.268149s boost 250.438468%
name | byte-code(s) | native-all(s) | native-all vs byte-code |
---|---|---|---|
bubble | 100.62 | 20.37 | 393.87% |
bubble-no-cons | 136.69 | 35.34 | 286.75% |
fibn | 121.39 | 67.13 | 80.82% |
fibn-rec | 109.00 | 38.57 | 182.60% |
fibn-tc | 103.86 | 24.52 | 323.48% |
inclist-tc | 101.63 | 2.02 | 4923.03% |
listlen-tc | 92.90 | 0.70 | 13224.35% |
inclist-no-type-hints | 124.36 | 5.93 | 1997.15% |
inclist-type-hints | 131.78 | 5.05 | 2511.98% |
nbody | 493.41 | 122.40 | 303.13% |
dhrystone | 552.13 | 161.88 | 241.07% |
linux
Summary results: bubble.el byte time: 145.803142s native time: 29.089018s boost 401.230884% bubble-no-cons.el byte time: 318.617107s native time: 38.999612s boost 716.975064% fibn.el byte time: 247.231776s native time: 75.364725s boost 228.047075% fibn-rec.el byte time: 103.741292s native time: 33.821895s boost 206.728206% fibn-tc.el byte time: 97.697834s native time: 31.647078s boost 208.710437% inclist-tc.el byte time: 130.386236s native time: 5.646812s boost 2209.023902% listlen-tc.el byte time: 119.750707s native time: 1.956367s boost 6021.075813% inclist-no-type-hints.el byte time: 320.839352s native time: 8.304878s boost 3763.264051% inclist-type-hints.el byte time: 322.535594s native time: 5.147577s boost 6165.775281% nbody.el byte time: 324.233682s native time: 87.086648s boost 272.311587% dhrystone.el byte time: 299.138452s native time: 171.954319s boost 73.963907% Total byte time: 737.046902s native time: 150.518523s boost 389.671895%
问题
稳定性
偶尔会segfault。
- M-x org-publish-all执行几次后报错,出现过segfault。
advice-add
override org-html-headline和org-html-section函数的时候没有生效。不编译eln就ok了。另外,用了package helpful后,查看编译为eln的函数,被override后,emacs的cpu会很高。而 C-h a 查看,看不见函数被override了。
ignore的函数是ok的。
在gdb的时候,只要执行2次 M-x org-publish-all,就会segfault。这。。。
cd src gdb emacs
- magit 在native compile后,commit的时候容易segfault。
功能
- helm
helm-buffers.el 编译为eln后,切换buffer时非常容易segfault。
Update: 在3月的某个commit之后,忽然就正常了,神奇啊。
- evil的visual mode按c-v,列编辑,按I插入不正常。
- undo-tree
发现undo-tree编译为eln后,doom-modeline在undo的时候,文件名不会从红色恢复为正常的颜色。
修改了evil-pkg.el中对undo-tree的版本依赖,改到2020年1月9日最新的v0.7.4版本,编译为eln后同样报错。
(advice-add #'undo-tree-undo-1 :after #'doom-modeline-update-buffer-file-name)
可能在函数执行后没有hook上去。
可能和加载顺序有关系,要advice-add的函数的eln必须要先加载。emacs自带的undo函数就是ok的,因为先加载了。
把editor调整到ui启动前。
- undo-tree
update - 2020.03.23
更新了不少东西。下面2个更新很好用。
make -j2 NATIVE_FAST_BOOT=1
NATIVE_FAST_BOOT编译选项打开后,只会编译生成pdump需要的elisp文件。大概只有10几个elisp文件会被编译为eln,编译速度只要10分钟左右。比原来快多了。如果经常改代码,dump id会变化,导致eln的生成目录名发生变化,每次都要全量编译740多个elisp文件会非常浪费时间。
comp-deferred-compilation
如果在make时使用了 NATIVE_FAST_BOOT=1 ,大部分emacs自带的elisp文件都不会被编译,可以在 early-init.el
或者 init.el
中加上
(setq comp-deferred-compilation t)
自动编译emacs自带的lexical scope的elisp文件和自己的elisp文件。
需要注意的是,elisp文件必须已经编译成elc文件,才会自动编译为eln。
在正常使用一段时间后,大部分常用功能的elisp文件都编译为eln了,建议关掉 comp-deferred-compilation
, 每次自动编译也有点浪费资源,而且有些文件不是lexical scope的,每次都尝试编译,却编译失败。另外每次会在mini buffer里看见的native-comp的buffer,有点烦。
update - 2020.04.22
更新到最新的代码后
(gdb) bt #0 0x00007ffe6d1e0aa3 in KERNELBASE!DebugBreak () from C:\WINDOWS\System32\KernelBase.dll #1 0x0000000400284727 in emacs_abort () at w32fns.c:10979 #2 0x00000004002ae187 in emacs_root_dir () at w32.c:3137 #3 0x0000000400161b02 in Fexpand_file_name (name=XIL(0x1a6304), default_directory=XIL(0)) at fileio.c:865 #4 0x0000000400165621 in check_file_access (file=XIL(0x1a6304), operation=XIL(0x61e0), amode=0) at fileio.c:2726 #5 0x00000004001656e9 in Ffile_exists_p (filename=XIL(0x1a6304)) at fileio.c:2751 #6 0x00000004001b16df in dump_do_dump_relocation (dump_base=70254592, reloc=...) at pdumper.c:5309 #7 0x00000004001b1986 in dump_do_all_dump_reloc_for_phase (header=0xbff450, dump_base=70254592, phase=LATE_RELOCS) at pdumper.c:5370 #8 0x00000004001b2207 in pdumper_load ( dump_filename=0x1a0a10 "E:\\workspace\\emacs_src\\emacs28\\emacs\\src\\emacs.pdmp", argv0=0xc13730 "E:\\workspace\\emacs_src\\emacs28\\emacs\\src\\emacs.exe", original_pwd=0x1a09b0 "E:/workspace/emacs_src/emacs28/emacs/src") at pdumper.c:5595 #9 0x0000000400113572 in load_pdump (argc=1, argv=0xc136b0, original_pwd=0x1a09b0 "E:/workspace/emacs_src/emacs28/emacs/src") at emacs.c:861 #10 0x0000000400113c19 in main (argc=1, argv=0xc136b0) at emacs.c:1069
最新的native-comp就编译不过去。在make时segfault了。奇怪啊,也没什么大动作怎么就失败了呢。gdb debug后发现在w32.c中获取环境变量
emacs_dir
,可能和native-comp分支有关系,export或者在bat文件中set emacs_dir就ok了。好在是一执行就segfault,可以直接gdb。
char * emacs_root_dir (void) { static char root_dir[MAX_UTF8_PATH]; const char *p; p = getenv ("emacs_dir"); if (p == NULL) emacs_abort (); filename_from_ansi (p, root_dir); root_dir[parse_root (root_dir, NULL)] = '\0'; dostounix_filename (root_dir); return root_dir; }
- src/comp.c改了不少,对应fixnum,自动判断是否支持unsigned long long类型。在mingw64中,long是32bit的。现在不用在gcc里改libgccjit的代码了,也用不着了。当然想在mingw64中用gccjit,libgccjit其他的代码还是要改的。