# Doxygen markup seems to cause py3 string parsing to sometimes fail

**URL:** <https://www.rubin.community/t/doxygen-markup-seems-to-cause-py3-string-parsing-to-sometimes-fail/1076>\
**Category:** Data Management\
**Created:** [August 23, 2016, 4:50pm UTC](https://www.rubin.community/t/doxygen-markup-seems-to-cause-py3-string-parsing-to-sometimes-fail/1076 "2016-08-23T16:50:18Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![natelust](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/natelust/32/358_2.png) [@natelust](https://www.rubin.community/u/natelust)\
**Post date:** [August 23, 2016, 4:50pm UTC](https://www.rubin.community/t/doxygen-markup-seems-to-cause-py3-string-parsing-to-sometimes-fail/1076/1 "2016-08-23T16:50:18Z")

</div>

When porting code to python 3, I came across a situation where the doxygen markup seems to tell python that we are trying to write a unicode literal, and importing the file fails with the error:

```python
Traceback (most recent call last):
  File "tests/testPsfIO.py", line 44, in <module>
    import lsst.meas.algorithms as algorithms
  File "/Users/nate/repos_lsst/meas_algorithms/python/lsst/meas/algorithms/ __init__.py", line 29, in <module>
    from .detection import *
  File "/Users/nate/repos_lsst/meas_algorithms/python/lsst/meas/algorithms/detection.py", line 211
    """
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2129-2130: truncated \uXXXX escape

```

The docstring itself can be seen on: [Github](https://github.com/lsst/meas_algorithms/blob/master/python/lsst/meas/algorithms/detection.py#L118). Prefixing the docstring with r (telling python a raw string is desired) fixes the problem. I am wary of embedding the escape character in the docstring itself, as it may interfere with doxygen generation. Does anyone have thoughts or preferences?

---

<div class="post-metadata">

**Author:** ![timj](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/timj/32/10_2.png) [@timj](https://www.rubin.community/u/timj)\
**Post date:** [August 23, 2016, 4:55pm UTC](https://www.rubin.community/t/doxygen-markup-seems-to-cause-py3-string-parsing-to-sometimes-fail/1076/2 "2016-08-23T16:55:35Z")

</div>

Specifically it’s the `\util` line in the string. That, obviously, isn’t allowed in a python 3 strings.

```auto
>>> type("\until")
  File "<stdin>", line 1
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 0-1: truncated \uXXXX escape

```

It looks like `r""` might be the only option until we implement [RFC-214](https://jira.lsstcorp.org/browse/RFC-214).

```auto
>>> r"\until"
'\\until'

```

---

<div class="post-metadata">

**Author:** ![jsick](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/jsick/32/672_2.png) [@jsick](https://www.rubin.community/u/jsick)\
**Post date:** [August 23, 2016, 4:56pm UTC](https://www.rubin.community/t/doxygen-markup-seems-to-cause-py3-string-parsing-to-sometimes-fail/1076/3 "2016-08-23T16:56:46Z")

</div>

I looked at what [PEP 257 recommends](https://www.python.org/dev/peps/pep-0257/#id15) and they say this:

> For consistency, always use “”“triple double quotes”“” around docstrings. Use r"““raw triple double quotes””" if you use any backslashes in your docstrings. For Unicode docstrings, use u"““Unicode triple-quoted strings””" .

So it seems that using raw strings for the current generation of doxygen-marked-up docstrings is the way to go. Numpydoc won’t have this issue (maybe for latex math in docstrings, but I don’t recall ever doing anything special in those cases).

---

<div class="post-metadata">

**Author:** ![natelust](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/natelust/32/358_2.png) [@natelust](https://www.rubin.community/u/natelust)\
**Post date:** [August 23, 2016, 5:00pm UTC](https://www.rubin.community/t/doxygen-markup-seems-to-cause-py3-string-parsing-to-sometimes-fail/1076/4 "2016-08-23T17:00:42Z")

</div>

That is what I figured, but I wanted to get other input before I just did it.

---

<div class="post-metadata">

**Author:** ![KSK](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ksk/32/393_2.png) [@KSK](https://www.rubin.community/u/KSK)\
**Post date:** [August 23, 2016, 5:06pm UTC](https://www.rubin.community/t/doxygen-markup-seems-to-cause-py3-string-parsing-to-sometimes-fail/1076/5 "2016-08-23T17:06:42Z")

</div>

I’m fairly surprised this hasn’t come up before. We use `\until` liberally in doxygen, so this will come up again.

---

<div class="post-metadata">

**Author:** ![rowen](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/rowen/32/52_2.png) [@rowen](https://www.rubin.community/u/rowen)\
**Post date:** [August 23, 2016, 5:10pm UTC](https://www.rubin.community/t/doxygen-markup-seems-to-cause-py3-string-parsing-to-sometimes-fail/1076/6 "2016-08-23T17:10:19Z")

</div>

Why not just use `@util` instead of `\util` and so on for all doxygen commands?

---

<div class="post-metadata">

**Author:** ![KSK](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ksk/32/393_2.png) [@KSK](https://www.rubin.community/u/KSK)\
**Post date:** [August 23, 2016, 5:11pm UTC](https://www.rubin.community/t/doxygen-markup-seems-to-cause-py3-string-parsing-to-sometimes-fail/1076/7 "2016-08-23T17:11:11Z")

</div>

Was that an intentional change in spelling? Is `\until` actually identical to `@until`?

---

<div class="post-metadata">

**Author:** ![jbosch](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/jbosch/32/12_2.png) [@jbosch](https://www.rubin.community/u/jbosch)\
**Post date:** [August 23, 2016, 5:11pm UTC](https://www.rubin.community/t/doxygen-markup-seems-to-cause-py3-string-parsing-to-sometimes-fail/1076/8 "2016-08-23T17:11:48Z")

</div>

> [@rowen](#):
>
> Why not just use @util instead of \util and so on for all doxygen commands?

+1

I tend to prefer “@” over "" in Doxygen universally (even in C++) for similar reasons - almost everything tries to interpret "", but only Doxygen (of the things we run on our source files) pays attention to “@”.

---

<div class="post-metadata">

**Author:** ![timj](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/timj/32/10_2.png) [@timj](https://www.rubin.community/u/timj)\
**Post date:** [August 23, 2016, 5:26pm UTC](https://www.rubin.community/t/doxygen-markup-seems-to-cause-py3-string-parsing-to-sometimes-fail/1076/9 "2016-08-23T17:26:12Z")

</div>

> [@KSK](#):
>
> I’m fairly surprised this hasn’t come up before.

Python 2, as is its wont, is far more relaxed about unicode in general so just silently ignores any unicode issues in strings. Python 3 cares deeply about unicode so complains.