Skip to content

Take 51degrees-cxx rc 865cee97 and generate the IP Intelligence wrapper again - #72

Draft
jwrosewell wants to merge 1 commit into
rcfrom
feature/take-cxx-rc-865cee97
Draft

jwrosewell wants to merge 1 commit into
rcfrom
feature/take-cxx-rc-865cee97

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Moves the cxx submodule from e866f6d3 (4.5.103) to the 51degrees-cxx rc commit 865cee97, so 51degrees-python takes the C library at the same commit as the other languages, and generates the SWIG wrappers again.

cxx commits taken

Every pull request merged into 51degrees-cxx rc between those two commits, from 51degrees-cxx:

  • #113 convert each value of the values collection to text once
  • #114 load only what the named properties read in HighPerformance, and keep Balanced the same whatever is named
  • #115 wait for a free file handle, and report reads that fail
  • #116 default Concurrency to the processor count, and size caches from it
  • #119 start the IP Intelligence C++ configuration from the default, which is Balanced
  • #120 give the data set dates from C, and check each is a valid date
  • #121 give the Hash match metrics and matched User-Agents from C
  • #122 scope the MetaData newobject features to MetaData, so ConfigIpi's getters are not freed
  • #123 list the evidence keys a data set accepts in C, and take the C++ engines' keys from there
  • #124 find a property's default value and write value text in C
  • #125 choose what the IP Intelligence data set holds by two configuration flags rather than the preset's shape
  • #126 list the Hash override and fixed evidence keys only when special evidence is processed
  • #127 refresh the Hash engine's evidence keys when its data is refreshed
  • #128 write the JavascriptHardwareProfile script in C, and call it from the C++ results
  • #129 give a property's type name from C, and take the C++ metadata's type names from there
  • #134 use the query value of special evidence before the cookie
  • #135 hold only the Hash values and strings the required properties read
  • #138 use every temporary folder given when deleting or finding temporary files, not only the first

The range also holds documentation, example and CI pull requests that do not change src (#96, #97, #98, #101, #102, #106, #108, #109, #111), pull requests #104 (free results and metadata in the examples and tests) and #105 (CMake package), and the merge of main into rc after release 4.5.103.

(git diff e866f6d 865cee97 -- src changes 65 files. Pull requests 118 and 139 merged into rc after 865cee97 and are not in this move.)

Wrappers

Python ships the two on-premise packages as source distributions and compiles the committed wrappers, so they are regenerated here.

  • SWIG 4.4.1, the version named in the header of the committed files, installed on a runner with pip install swig==4.4.1.
  • The same command PreBuild.sh runs, swig -c++ -python hash_python.i and swig -c++ -python ipi_python.i, from the package source folder after the cxx source is staged into it as setup.py does.
  • Check that the method reproduces the committed files: run against e866f6d3, the same command gave hash_python_wrap.cxx, DeviceDetectionHashEngineModule.py, ipi_python_wrap.cxx and IpIntelligenceEngineModule.py byte for byte the same as the ones committed (empty git status and git diff). Run: https://github.com/51Degrees/51degrees-python/actions/runs/42673222229
  • The same run against 865cee97 changed only ipi_python_wrap.cxx, in four lines. The ConfigIpiSwig getters getComponents, getProperties, getValues and getProfiles returned a CollectionConfig with SWIG_POINTER_OWN and now return it with 0, because #122 scoped the %newobject directives to MetaData and the configuration owns those references.
  • hash_python_wrap.cxx, DeviceDetectionHashEngineModule.py and IpIntelligenceEngineModule.py come out identical, so they are not in the diff.

Build files

device-detection/.../setup.py and ip-intelligence/.../setup.py find the cxx C and C++ sources by glob over common, dd or ipi (and dd/hash or ipi/graph), and MANIFEST.in includes *.h, *.hpp and *.i recursively. Between the two commits one source file was added, src/common/date.c, and none was renamed or removed. The glob picks the new file up, so neither setup.py changes.

Options

setLoadRequiredPropertiesOnly, setDefaultValuesInMemory and the concurrency default are new in the C++ configuration. The existing profile calls in devicedetection_onpremise.py and ipintelligence_onpremise.py (setLowMemory, setMaxPerformance, setBalanced, setBalancedTemp, setHighPerformance, setConcurrency and the rest) are unchanged in the C++ headers, and no new Python API is added.

Verification

The full build does not pass yet, and the cause is in the C library, not in this repository's wrappers.

  • Full build, run 42673821633, head f804c54b (every Setting 'PullRequestSha' line matches). The first attempt failed before creating any build job (a "workflow file issue" with no detail, which cleared on a rerun). On the rerun, Windows_Python_3.13 failed in this job and the matrix then cancelled the other 14 jobs, so Ubuntu and macOS never reported a result. The UTM link lint check passed on a rerun after the same kind of start-up failure.
  • The failure is the linker, for both on-premise packages: error LNK2001: unresolved external symbol "public: __cdecl FiftyoneDegrees::Common::Date::Date(void)", and the same for Date::getDay, getMonth, getYear and the Date(const fiftyone_degrees_date_t *) constructor.
  • Cause: #120 added src/common/date.c beside the existing src/common/Date.cpp. The names differ only in case. This repository's setup.py compiles the C files and the C++ files into the same build folder, so on a case-insensitive file system (Windows, and macOS by default) both write Date.obj/date.obj to one path. The CI builds each package twice in one tree (python setup.py build_clib build_ext --inplace, then pip install .), and the second build sees the Date.obj the C compile left behind as newer than Date.cpp, skips compiling Date.cpp, and so cpplib.lib holds no C++ Date.
  • Checked on a runner (Windows, Python 3.13, the same two commands in the same order), on a temporary branch since deleted. Unchanged, the build fails with those five LNK2001 errors. With date.c renamed to date_c.c in the cxx working copy only, the same commands pass. A single pip install in a fresh tree passes either way, which is why the problem only appears in the CI's two-step order.
  • Not changed here. Renaming one of the two files belongs in 51degrees-cxx (it already names the C++ side of a C file with a _pp suffix, as in string_pp.cpp), and it affects every language that compiles the sources into one folder. Python was not worked around in setup.py.
  • macOS was not tested (its runners were queued). It has the same case-insensitive default, so I expect the same failure, but that is not checked.

No other job reported a failure, but the other 14 were cancelled before they finished, so they prove nothing. The SWIG wrapper check above ran on Linux and the Windows build reached the link step with the regenerated hash_python_wrap.cxx and ipi_python_wrap.cxx compiling cleanly.

What remains

  • The C library needs the Date.cpp and date.c name clash fixed before this builds on Windows or macOS (see Verification). The draft then needs a new full build.
  • 51degrees-cxx rc has moved on since 865cee97 (to d71ad22f at 18:21 UTC, with 118 and 139), and more drafts are still to merge. This draft will move again to the final commit.
  • Release candidate #67 is untouched.

…er again

Moves the cxx submodule from e866f6d3, 4.5.103, to the 51degrees-cxx rc
commit 865cee97.

Generated hash_python_wrap.cxx, DeviceDetectionHashEngineModule.py,
ipi_python_wrap.cxx and IpIntelligenceEngineModule.py again with SWIG
4.4.1, the version in the committed files, from the staged source the
way PreBuild.sh does. Run against e866f6d3 the same command gives
files byte for byte the same as the committed ones, so every
difference comes from cxx.

Only ipi_python_wrap.cxx changes. The four ConfigIpiSwig getters that
return a CollectionConfig no longer pass SWIG_POINTER_OWN, because the
cxx interface now scopes its newobject directives to MetaData and the
configuration keeps ownership of those references. The Device
Detection wrapper and both module files are unchanged.

The setup.py files list the cxx sources by glob, so the new
src/common/date.c is picked up with no change there.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant